Today's Paper: Learning to Generate Long-term Future via Hierarchical Prediction

Great demonstrations of powerful combination of recent deep neural networks applications - stacked hourglass networks for human pose estimation [Newell 2016] and visual analogy learning [Reed 2015], enabled amazing results in future video prediction.

Learning to Generate Long-term Future via Hierarchical Prediction, ICML 2017 (to appear) [Arxiv]

Great demo videos are available online.

Abstract:

We propose a hierarchical approach for making long-term predictions of future frames. To avoid inherent compounding errors in recursive pixel-level prediction, we propose to first estimate high-level structure in the input frames, then predict how that structure evolves in the future, and finally by observing a single frame from the past and the predicted high-level structure, we construct the future frames without having to observe any of the pixel-level predictions. Long-term video prediction is difficult to perform by recurrently observing the predicted frames because the small errors in pixel space exponentially amplify as predictions are made deeper into the future. Our approach prevents pixel-level error propagation from happening by removing the need to observe the predicted frames. Our model is built with a combination of LSTM and analogy based encoder-decoder convolutional neural networks, which independently predict the video structure and generate the future frames, respectively. In experiments, our model is evaluated on the Human3.6M and Penn Action datasets on the task of long-term pixel-level video prediction of humans performing actions and demonstrate significantly better results than the state-of-the-art.

A one shot illustration of their approach can be:

(A picture in the paper, with my annotation of leveraged existing work references.)

ROS Subscriber may be immediately unsubscribed

A caveat for programming in C++ for ROS (Robot Operating System).  The core ROS architecture is of the pub-sub messaging framework, where you can publish on a topic to send messages and subscribe on a topic to receive messages, among ROS nodes.

Both publishing:

ros::NodeHandle nh;
ros::Publisher pub = nh.advertise("some_topic", ..);pub.publish(message);

and subscribing:

ros::Subscriber sub = nh.subscribe("some_topic", .., callback);

are fairly simple to write.  (The callback can be either a function or method, to handle the message received.)

You must hold the returned subscriber object till the end of running your application.  It is a pitfall you can simply ignore it as follows:

nh.subscribe("some_topic", .., callback);

This might result in the deletion of the subscriber object, calling its destructor to unsubscribe from the topic...  You won't have any callback, then.  ;-<

Set up ROS Kinetic in macOS

Installation

The official wiki] gives you an extensive (it shouldn't be ideally, though) explanation in installing ROS Kinetic in macOS (f.a.k.a. MacOS X).  Its Trouble Shooting section is mandatory (but not sufficient) to read through.

One of the largest pitfalls in the process is with Qt5 and its python plugin.  Qt is a popular cross-platform UI development framework, on which many applications depend.  Installing `qt` with Homebrew provides Qt4 but we need Qt5 for ROS, which may conflict with Qt4 at least in the Homebrew level.

In my environment to have `Desktop-Full Install` (`Desktop Install (recommended)` might be a lot simpler), at least following was necessary to be set up:

$ brew install --force qt5
$ brew link --force qt5
$ brew install pyqt5 --with-python
$ ln -s /usr/local/Cellar/qt/5.8.0_2/mkspecs /usr/local/mkspecs
$ ln -s /usr/local/Cellar/qt/5.8.0_2/plugins /usr/local/plugins
$ ln -s /usr/local/share/sip/Qt5 /usr/local/share/sip/PyQt5

as well as:

$ brew tap homebrew/science
$ brew install python ; brew linkapps python
$ brew install --force opencv3 --with-contrib ...
$ echo /usr/local/opt/opencv3/lib/python2.7/site-packages >> /usr/local/lib/python2.7/site-packages/opencv3.pth
$ brew install cmake sip poco eigen qhull pcl libogg theora gazebo8

See the Trouble Shooting section and an issue raised in Homebrew GitHub for more details.  Once prepared these prerequisites, you may go with the `rosdep` step in the wiki but with a `--skip-keys` option:

$ rosdep install --from-paths src --ignore-src --rosdistro kinetic -y \
        --skip-keys "libqt5-core libqt5-gui libqt5-opengl libqt5-opengl-dev libqt5-widgets qt5-qmake qtbase5-dev pyqt5"

to allow the installer to skip checking them as you are sure they are there.  See a Q&A entry for more details.

You need to avoid the `string.h` issue in the system, by specifying `-DCMAKE_FIND_FRAMEWORK=LAST` for `catkin_make_isolated` to use it from the standard library.  See an issue in the `ros/rosdistro` GibHub for the solution.  Also, according to a workaround for Homebrew Python Segfault issues] it might be better to specify PYTHON_INCLUDE_DIR and PYTHON_LIBRARY variables to point to Homebrew's.

$ ./src/catkin/bin/catkin_make_isolated --install \
    -DCMAKE_FIND_FRAMEWORK=LAST -DCMAKE_BUILD_TYPE=Release \
    -DPYTHON_INCLUDE_DIR="/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Headers" \
    -DPYTHON_LIBRARY="/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/libpython2.7.dylib"

Some fix was needed for me.  For `src/image_pipeline/image_view/CMakeLists.txt`:

#find_package(OpenCV REQUIRED)
find_package(OpenCV HINTS /usr/local/Cellar/opencv3/3.2.0/ REQUIRED)

Maybe optional but for `src/class_loader/CMakeLists.txt`:

#find_package(Poco REQUIRED COMPONENTS Foundation)
find_package(Poco HINTS /usr/local/Cellar/poco/1.7.8_1 REQUIRED COMPONENTS Foundation)

Do not forget to purge components under `*_isolated/` if you have accidentally failed to build.

To have it consistent with the Ubuntu convention, instead:

$ ./src/catkin/bin/catkin_make_isolated --install \
    --install-space /opt/ros/kinetic \
    ...

Fix rviz with a workaround

With Qt5, rviz causes Segmentation Fault due to an improper parameter passed to Ogre's createRenderWindow().  This is due to rviz's dependency on `QT_MAC_USE_COCOA`, used to be defined in Qt4, but no more in Qt5 probably because Qt5 only works with Cocoa but not with Carbon.  A workaround can be found in an issue raised in rviz.

Edit `src/rviz/src/rviz/ogre_helpers/render_system.cpp` to have:

// This is required for QT_MAC_USE_COCOA to be set
#include
// workaround as Qt5 doesn't set it properly
#define QT_MAC_USE_COCOA

Then rebuild rviz:

$ rm -rf devel_isolated/rviz build_isolated/rviz
$ ./src/catkin/bin/catkin_make_isolated --install --from-pkg rviz -DCMAKE_BUILD_TYPE=Release -DCMAKE_FIND_FRAMEWORK=LAST

***

Tips in Trial and Errors

To resume from intermediate, where you failed to build and fixed (such as installing dependencies):

$ ./src/catkin/bin/catkin_make_isolated --install \
    --from-pkg

Other References

http://stackoverflow.com/questions/21064128/cant-get-opencv-to-work-on-osx-using-cmake
http://answers.ros.org/question/95056/building-rosconsole-osx-109/

# Running roscore and rviz

    $ pip install defusedxml
    $ brew install ogre1.9

First thing is to run `roscore`, which is a collection of nodes and programs that are pre-requisites of a ROS-based system.

    $ source /opt/ros/kinetic/setup.bash
    $ export ROS_HOSTNAME=localhost
    $ export ROS_MASTER_URI=http://localhost:11311
    $ roscore

For setting up networks, see an official Wiki.
You may see something like following:

    $ roscore
    ... logging to /Users/mich/.ros/log/c1234e21-1eb7-11e7-a096-c4b301ccf3ff/roslaunch-mich-mbp2.local-62976.log
    Checking log directory for disk usage. This may take awhile.
    Press Ctrl-C to interrupt
 
    started roslaunch server http://localhost:55042/
    ros_comm version 1.12.7
 
 
    SUMMARY
    ========
 
    PARAMETERS
     * /rosdistro: kinetic
     * /rosversion: 1.12.7
 
    NODES
 
    auto-starting new master
    process[master]: started with pid [62983]
    ROS_MASTER_URI=http://localhost:11311/
 
    setting /run_id to c1234e21-1eb7-11e7-a096-c4b301ccf3ff
    process[rosout-1]: started with pid [62986]
    started core service [/rosout]

In another Terminal shell (tab/window),

    $ source /opt/ros/kinetic/setup.bash
    $ rviz rviz


It was supposed to be:

    $ rosrun rviz rviz

but `rosrun` won't work for now.

See an official Wiki page for more details.

Set up OpenCV for Python in MacOS X

Install/update Xcode from App Store and launch it to pass through terms agreement prompts.

Install Xcode Command Line Tools for easing later processes.

xcode-select --install

c.f. Stack Overflow thread

Install OpenCV.

$ brew install gcc
$ brew install opencv3

Now install python modules for OpenCV.

$ pip install cv2

You have to provide a shared library for it.  This is the key step. 

$ pushed /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages
$ ln -s /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cv2.so
$ popd

Test if it works.

$ python
>>> import cv2
>>>  

It should finish without showing any error.

If this is for your virtualenv, do similarly in your venv directory.

Linked Data Basic Profile Distilled

Here are key practices we need to follow according to the Linked Data Basic Profile 1.0.

Use standard HTTP methods for touching resources.
As well as SPARQL, a resource server should provide HTTP methods of POST or PUT, DELETE, PUT or PATCH, and GET for creating, deleting, updating, and fetching resources. For update-collision tolerance, clients should use HTTP If-Match and ETags for updating resources (with PUT or PATCH).

Use RDF/XML for representing any resource indicated by and requested through a URI.

To be continued.

Essence of Linked Data Basic Profile

In April, the "Linked Data Basic Profile 1.0" was published as a W3C member submission, lead by IBM Rational but involving other vendors such as Oracle and RedHat as well as SemanticWeb.com. It specifies practical rules that clarify or extend Tim Berners-Lee's basic rules (Use URIs as names for things, provided with info in RDF* or through SPARQL accessible through HTTP for the URIs. Also, these info should include other URIs to discover more.)

The specification try to answer the rest of things practically inevitable to provide better Linked Data. The questions answered are:

• How do we create a resource, or what do we POST to?
• Where can we get the list of existing resources?
• Which vocabulary and media types do we use?
• How do we split the information into pages when it gets big?
• How do we specify ordering?

For that, the set of rules focus on the following concepts:

• Basic Profile Resources (BPR) - HTTP and RDF techniques to use to read/write liked data
• Basic Profile Containers (BPC) - a BPR for POST to create new things and GET to find existing
• Paging - a mechanism to get the content of a BPC in chunks
• Ordering - a mechanism to specify the order of a sorted BPC

For more information, a developerWorks article and a paper and its presentation at WWW 2012 workshop on Linked Data on the Web (LDOW 2012) help to understand the motivation behind the spec. Also, there is a collection of use cases and requirements on which the spec was built.

Today's Paper: WSLA for Crowds

A promising insight for crowds (crowdsourcing) as a service, which should/will have a standardized interface for service requests among various crowds platform providers in the near future.

• Khazankin, et al., QoS-Based Task Scheduling in Crowdsourcing Environments, ICSOC 2011. [EE]

Abstract:

Crowdsourcing has emerged as an important paradigm in human-problem solving techniques on the Web. One application of crowdsourcing is to outsource certain tasks to the crowd that are difficult to implement as solutions based on software services only. Another benefit of crowdsourcing is the on-demand allocation of a flexible workforce. Businesses may outsource certain tasks to the crowd based on workload variations. The paper addresses the monitoring of crowd members’ characteristics and the effective use of monitored data to improve the quality of work. Here we propose the extensions of standards such as Web Service Level Agreement (WSLA) to settle quality guarantees between crowd consumers and the crowdsourcing platform. Based on negotiated agreements, we provide a skill-based crowd scheduling algorithm. We evaluate our approach through simulations.