Questions and Answers of Robotics in ROS – Week 4

Recommended readings/lectures: ROS wiki on Navigation stack, Udacity’s Artificial Intelligence for Robotics (brief discussions of Unit 1, 4), AI page for Wikipedia on paradigms and approaches, ROS Wiki on data types

One of the things that got me involved in robotics was the fact that a good robot have to be a sufficiently intelligent agent – capable of sensing the “right” parts of the environment (computational cost involved in sensing requires that every agent be discerning) and acting real-time based on those decisions.

Thus, the challenges of robotics involve those two questions – how and what should the robot sense in the environment, and how can the robot act based on that data in a way that could mimic the future-planning and strategic agents like us?

First, let us address the first question – what exactly is involved in the robot’s perception of the environment, or phrased more concretely, why is making a perceptive robot difficult?

One cause of difficulty is the inherent uncertainty that comes with perception. Whether it is caused by the environment or the sensors, the uncertainty of perception necessitates that the robots process what they perceive, and discern the probable reality based on their previous perceptions. For example, we humans can discern that the visual illusions are illusions by analyzing the senses and declaring them to be false, based on our limited understanding of our visual system.

The idea of taking bunch of uncertain things to unify them into a more certain conclusion is an essential idea of robotic perception, and it inherently involves probabilities.

Similarly, another cause of difficulty is the difficulty of integrating multiple sensory data to form one (sometimes multiple) coherent model. Effective perceiving agents have multiple kinds of senses. For example, humans have senses of vision, tactile, etc – many of which informing another sense (e.g. food tastes better if you can smell them). Likewise, robots have a wide array of disparate sensors – ranging from LIDAR (distance sensor that uses laser), infrared sensor, wheel encoders (keeps track of wheel rotations) – and they must be able to utilize all of them to deliberate an action (not necessarily a cohesive action, some theory of AI/robotics argue for the possibility of separate sensors and actuators (limbs of robots) within one robot – an arm acting separately from the body, for instance).

Not only can the type of sensor data be different, but the sensory data could be coming from different parts of the robot – a camera on the left arm and a right arm, for instance. Such scenario gives way to its own sets of problems: how do we “sync” up the changes between the two cameras to get the one coherent view of the room? What does the camera on the arms inform us of the situation the robot is in (the body, etc)?

After the issue of the senses and perception, we are then confronted with the issue of acting and planning (for the future). Indeed, any robots worth their money must be able to plan for the future in some way – not only for the effectiveness of the action, but for the intelligence of the robot.

First question we can think of is the issue of producing good models of reality based on the perceptions. Yes, the perception and sensor data may be in, but they are of no use to us (and the robot) if they can’t be modeled and “understood” properly. For example, if the LIDAR returns to us a 2D vector of points as perceived relative to the sensor, how or for what can we use the data?

Another question is a practical one of making real-time decisions based on the model. Even if the robot have the best model in the world, yet if there is no computationally practical way of acting real-time according to the model, then the robot would be terrible and the model useless. While this practical concern had been well mitigated by the Moore’s Law, it is still and will be a concern in the future.


These issues are thankfully addressed to some extent in the ROS library, particular in the part of the library called the navigation stack. Navigation “stack” features many nodes that each deal with an issue.

The navigation stack largely addresses these issues in the context of mobile, autonomous robots. As seen in the diagram below, nodes that deal with sensor data and perception goes into the box part of the diagram, which largely acts as a brain (more specifically, a state machine) that gives out a command for action (cmd_vel and path for the future). With that cmd_vel, the programmers would tell the robot how to move, writing a motor controller program that can receive the commands in terms of cmd_vel.

In the upcoming days, the nodes you’ll interact with will be nodes that will serve to be basic processor for the sensors – taking in perception and providing the data in a specific format.

overview_tf

Callbacks and Synchronicity – Week 3

Robotics and its software differ from a typical software in its two main requirements.

The first requirement is that robot software must react differently based on environmental changes and conditions – often modeled by a state machine and its states (see here, but at its simplest, it’s just if’s in a while loop).

For example, should the robot not enter into a recovery state after ramming into some obstacle, rather than continuing to pummel through? Even if all the visuals and sensor data may be the same, the robot must act differently in a recovery state than the normal state.

Another requirement is that the robot (the state machine above or something else) needs to be able to handle inputs and requests from multiple sources – from the sensor data, requested actions, and such. Here, the complexity emerges, and ROS is handed the duties of a typical operating system – managing multiple programs at once.

ROS, in its distributed approach, does not have THE state machine, but rather leaves it to each node to determine its own state and resolve the potential conflicts (an approach supported by recent neuroscience advances).

So, when designing nodes for ROS, one must be aware of these requirements.

But first of all, let us first discuss the service tool, which is a synchronous way to establish one-to-one communication, which means that there is an order to the communications. The order is the following: A node (client) would send a request to another node (server), possibly waiting for a reply as the server processes the request.

Where in the code do the waiting and processing happen? The waiting happens at spin() and spinOnce(), while the processing happens in the callback.

Each node, when they receive messages through topics or service, does not immediately process them, as they very well might be doing something more important. Instead, the messages are held in the queue (a line essentially) and processed as a batch during the spin() commands.

So what is the difference between spinOnce() and spin()spin() is essentially spinOnce() called in a loop – once the node runs spin(), the node will now only deal with the callbacks. By comparison, spinOnce() allows the node to process the callbacks at the designated time “once”.

So when should you use each? You should use spinOnce() if you have a code not part of the callback that constantly needs to run and spin() if the server only needs to deal with callbacks from then on. Also, with spinOnce(), you need to be aware of the time that will take to process the tasks between spinOnce(), making sure that the tasks will not prevent callback from activating too late.

And for the services in general, you need to also consider the time that the service will take, making sure it doesn’t occupy the server for too long. Otherwise, the server will not be able to react quickly to the inputs from other nodes – perhaps a command that is requesting for the robot to freeze immediately. Thus, it is generally recommended that services be simple, short requests.

However, you would need the ability to command a robot to do a more complicated task – or even allow for multitasking of tasks. For those requirements, there is the action lib. In contrast to service, it is – and must be – an asynchronous way to communicate one-to-one. 

Why the must? The actions must be asynchronous, as for actions with a long-time frame (~seconds), the state of the robot might change, with actions needed to be interrupted and continued later. So actions are asynchronous because the execution of action is not fixed – unlike services, which can’t be interrupted.

While the exact details of both service and action is in the respective tutorials, these differing goals for services and actions must be accounted for when making servers (nodes) for each of them – asking if the service or action would be more appropriate at this point.

Assignment: Implement a client/service, in which client sends in a service to the server to change the state of the server in some way.

 

 

 

 

 

 

 

Intricacies with C++ and C++ OOP – Week 3

Note: Both C++ and OOP in C++ are enormous topics, so the mechanisms/concepts discussed here will be a small, limited subset of the subject. Also, general programming/OOP concepts are not addressed.

General Intricacies of C++

  • C++ is wholly compatible with C, which means that you can compile and run C programs as C++ programs, with some changes with the library/header names. The C libraries in C++ are prefixed with c, i.e. the string.h is cstring in C++.
  • Previously considered “lacking” in conveniences previously, C++ had “recently” (2014) gone through a major standard change called C++11. Features added are..
    • Standard Template Library (often abbreviated STL). STL allows for a generic (supporting multiple classes) data structure like vectors (which are arrays (linked-list implementation) with dynamic (changeable) size), queues, stacks, etc.
      • STL uses things like:
        • Iterators (essentially indexes)
        • auto (deduces type, often used for iterators due to their long name), e.g. auto i = numVector.begin(); assigns the iterator type to i.
      • Templates are often specified like template<classname T>. So, if you see a function specified with such brackets, you can infer that the function is designed to be used with multiple classes.
    • There is now nullptr to indicate null pointers (NULL is known to cause issues in error handling)
  • Nor do C++ have many of the practical features implemented, so many/most C++ programmers complement the language with a library called Boost, which provides features for Serialization, I/O streams (often used for parsing input/data), and Math.
  • C++ print to standard out using std::cout << "string" << endl; and standard error std::cerr << "string" << endl;, eschewing commonly used print statement.
  • Like C, the default in C++ is that values passed into functions are copied, rather than via reference like Java – which can cause performance issues. That is the reason why so many functions have & notation next to the parameters, indicating that the parameters are passed via “reference”, or more accurately, via pointers (unfortunately, I can’t go into pointers here).
  • You can have default arguments/paramters in C++.
  • C++ have no garbage collection, which means that you have to destroy everything that has been created dynamically with the new operator with delete. Use of C’s malloc and free is discouraged.

OOP intricacies with C++

  • C++ is more flexible with overriding things in classes in general, allowing the overriding of things operators (+, <<), destructors (things that destroy class constructor).
  • C++ have something called namespaces (which is also used in ROS) to organize things variables and functions without the structure of a class.
    • Namespaces are used to remediate the problem of global variables. Global variables are evil, as you could unintentionally call global variables from other places of the project without knowing – i.e. if you declare a global variable with a generic name of speed, the variable can now be used/changed everywhere.
      • Alternatives to global variables are enums.
    • Namespaces are called name::variable and declared namespace name { auto variable = 0; }
      • Class functions and variables, too, are called as  class::variable.
  • You indicate public, protected, private parts of class using notations public:, private:, protected:
  • Objects/Instances of the class will be destroyed once the scope of the instance is exited, e.g. with void init() { Dog d = Dog(); }, Dog d will be destroyed once init function ends. You can avoid this using the new operator discussed earlier.

Feel free to inform me if there is any part of C++ code that is confusing – I’ll add onto the list.

ROS Workspace and Packages – Week II

Refer to: Martinez’s Learning ROS Chap. 2, Gentle Intro to ROS 2.4 ~ 4

Note: We have ROS version Jade in both the computer and the laptop – there shouldn’t be too much differences yet, but nonetheless I should have informed you all earlier.

The distributed nature of ROS development (allowing different developers to contribute to ROS) required ROS to adapt to an effective package system. Without such system, the difficulty involved with integration and compilation of external code would have prevented ROS from taking off.

Accordingly, ROS has a convenient (relatively) package system called Catkin, and ROS’s promotion of Github allows for an easy integration with the external code. Catkin is a variant of the Makefile system – a program/format that automates a large part of the compiling task. With Catkin and Make systems, you just need to specify the C++/Python files with the respective libraries in text files. For Catkin system in ROS, the text file used is called CMakeLists.txt and package.xml, which are in each ROS packages.

ROS packages, each represented by the folders in Catkin Workspace (catkin_ws), contains the compiling instructions (the two text files above), as well as the source files of the package. Furthermore, they could contain msg files (for specifying topic), launch files (files that can be used to launch multiple programs at once), and so on – containing every code or configuration that is needed to do a specific task. Given that ROS packages largely exist for organization purposes, it is considered a good practice to only specify one function to each of the packages.

With all the files set up in the packages, the typical workflow goes as follows:

  1. First, I type out the program in C++ or Python and place it in the correct folder (src for C++, scripts for Python is recommended). Note that I am talking about src and scripts folder inside the package folder! So like ~/catkin_ws/package/src/ ...
  2. I specify the files and the libraries they need in CMakeLists.txt so that Catkin system may find them and compile/turn files executable.
  3. In the catkin_ws folder, I run source devel/setup.bash, which sets up the Catkin system so that it may be able to detect all the packages.
  4. I run catkin_make in the catkin_ws folder – which causes the Catkin system to start compiling all the packages in the src directory of catkin_ws. Also, msg and srv files are made into C++ header files, allowing you to import the topic messages into the program (by header files, just know for now that it would provide specification/type/function that the message requires to function).
  5. The compiled programs (if compilation was successful) end up in the build directory, able to be executed via rosrun <package_name> <file_name>.
  6. As a side note, you can disable compilation of certain packages by making an empty file called CATKIN_IGNORE in the package folder. To easily do that in the directory, use the command touch CATKIN_IGNORE.

Here is the workflow I use for integrating another package from Github into my project:

  1. I find the Github page for the package I want to download – let’s say https://github.com/ros/ros_tutorials – usually the Github page is at the top of the ROS wiki.
  2. I enter ~/catkin_ws/src.
  3. I make sure that the package is for the right ROS version (in our case, Jade) and I use the command git clone https://github.com/ros/ros_tutorials, which copies over the contents of the git repository into the ~/catkin_ws/src/ros_tutorials folder.
  4. EXTRA: I remove the .git file in the ros_tutorials (not one inside the src folder!!). .git file contains the git information (file changes, etc) and indicates that the folder with it is a separate git repository. To integrate the package into our own repository, we need to remove it. If not, you can keep it in.

Do read Martinez’s Learning ROS Chapter 2 – it has a detailed information about ROS package system.

Task: Download turtle_sim package and make a publisher that makes the turtle move in place in “circles”.

 

Introduction to Git and Github – Week 2

 

When discussing tools used for the open-source programs, git and Github are the two tools that inevitably come up.

What is git and Github and why are they so prominent in the open-source world?

Git is a distributed version control software, which means that you can access and modify your code on your own computer in the version of the program you see fit.

By contrast, the alternative to git is a centralized version control software, which requires you to connect to a server with the code to make changes to the program.

As you might guess, this distributed approach not only allows greater flexibility of how you could work but also leads to fewer conflicts that may arise from people sharing the same “workspace”.

Of course, git still involves the main code base (referred to as the master repository). Github is a free and open platform that is often used to host this repository, along with other versions of the code known as “branches”.

If you are confused at this point, have no fear – while git is somewhat infamous for its difficult user interface, there are only a few commands and workflows that you need to know to make use of git (as with all tools). I, for one, just know about and use ~10 commands.

Instead of attempting to learn about all the git commands (of which there are many), I would recommend that one prioritize learning about the git workflow (how to use the commands effectively).

Here are the three links I recommend, in order:

Essential Git Commands

Git Workflow

How to Use Github

As always, do inform me if the coverage of the materials are too brief/long/quick/slow – I’ll try to adjust accordingly.

How I Learned to Stop Worrying and Love the Terminal

zGiven that most of you will access ROS through the terminal (the -X next to ssh indicates that you can try to run GUI tools, but the speed is excessively slow), here are some terminal commands and tools that will make working in the terminal a lot easier.

tmux 

Tmux is a terminal multiplexer, which means that you can run multiple terminal shells in one window. Type tmux  into the terminal, and it should take you a nearly identical terminal window.

Commands are

  • Ctrl-b + % – Split the terminal vertically into two panes
  • Ctrl-b + " – Split the terminal horizontally into two panes
  • Ctrl-b (with continued press) + arrow_key – Resize the pane
  • Ctrl-b + arrow_key – Change pane selection
  • Ctrl-b + x – terminal the current selected pane
  • Ctrl-b + [ – Read mode (ESC and move cursor to escape)

vim (or nano)

Vim is a text editor that requires the users to use key commands. You could either type vimtutor for practice (which will take about 10 minutes), or google for basic vim commands. You only really use ~10 commands, so I think it’s worth learning vim.
I highly recommend you go through vimtutor multiple times.
Currently, there is a file explorer pane in vim (called netrw) – you can switch between panes using Ctrl-W + Arrow_keys  and terminate each pane using Esc and typing :wq (for editor) and :q (file explorer).
Terminal commands

To redirect the terminal output to a file (so you can read the error output)

some_command >file.log 2>&1 

Run a command in the background:

some_command & 

Move out of current folder

cd ..

How to run a command every terminal launch

Modify ~/.bashrc (e.g. vim ~/.bashrc) and put commands at the bottom.

~/.bashrc is essentially a shell (bash) command that runs when you first open the terminal, which means that you can put setup commands in ~/.bashrc so that you would not have to type them over and over again.

In ~/.bashrc, you can put ~/catkin_ws/devel/setup.bash (which is within ROS workspace) as one of the setup files to execute at launch. Whatsetup.bash does is that it sets up the path variables for ROS, allowing commands like roscd to find where the directory with packages is.

For those who are not so familiar with the terminal language, a “shell” is the terminal – a way for the user to interact with the operating system (Linux, in this case).

ROS commands

roscd <package_name> <folder_name>

rosed <pacakge_name> <file_name> – Opens file for editing

Commands like roscd (ros + cd) and rosed (ros + ed), they are essentially shortcuts for cd and ed to edit ROS package files quickly.

 

Independently Learning Robotics

Of course, learning ROS without the robotics knowledge is useless. So how can a person independently learn robotics? While I will cover or point to some of the ideas of robotics, here are the resources you should use.

As I recommended before, I think Artificial Intelligence for Robotics class in Udacity is really helpful in developing the understanding of the essence of robotics (if I had any sense of it at all). I found that standard robotics class or courseware focuses too much on the mechanics of the robot rather than its intelligence (joints and angles everywhere!). Even if you are not familiar with the programming part of the Udacity course, skimming through the idea part of the course is, I think, a good idea.

ROS wiki itself is a good resource. Go through each of the packages in our code repository and read what each package do – it is easier than you would think.

ROS have truly transformed the job of the programmers in the robotics field, as much of the robot functions are implemented in the aggregate of the ROS libraries. Just as programmers don’t necessarily need to understand the string representation in a language to use it, robotic programmers do not need to know the implementation details of each and every robot parts – just of its utility.

Independently Learning ROS

For people who wants to get ahead on the Independent Study (be independent, in others words), I would recommend that you cover two texts simultaneously:

Lentin Joseph’s Mastering ROS and

Gentle Intro to ROS by Jason M. O’Kane.

Mastering ROS covers the practical applications of ROS (essentially building the same differential-drive (two motorized wheel) robot), and Gentle Intro to ROS covers the technical details of ROS.

Going through the applications and occasionally looking at the technical details of ROS is, I think, the way to go.

Introduction to Linux- Week 1

As of this moment, ROS can only run on the operating system called Linux (specifically on a distribution called Ubuntu), so it is essential to be familiar, or at least comfortable with Linux to be able to use ROS effectively.

Linux, in its entirety, is an enormous topic, so let me just cover the enough materials to make you dangerous.

You will largely interact with ROS through a program called terminal, which can be launched with keys CTRL-ALT-T.

Terminal allows you to interact with the OS through text commands, as determined by the ‘shell’ you are running. Most Linux distributions, as well as Ubuntu, runs a shell called bash shell. Type help into the terminal to see the list of commands (which isn’t that many).

The bash commands are programming language themselves and can be used to write simple programs that are often suffixed with .sh. For example, many program installations in Linux is done by running ./install.sh, which usually contain the bash commands is needed for installation of the pogram.

The dot front of the slash indicates that the file is in the current directory, which you can change by typing cd.

All of the linux commands run in such format: <command> <arg 1> <arg 2> .... <arg n>. For example, I can run

Other commands or marks to know is

..  indicates the directory above the current one.

~/  indicates the home directory of the user, e.g. ~/Downloads

ls   lists all directory and files in the current directory

chmod +x <file name allows the file to be executable.

ssh allows you to connect to a remote Linux machine using ssh protocol

Other commands can be referenced by a simple Google search.

See:

-https://www.howtoforge.com/useful_linux_commands

-http://tldp.org/LDP/abs/html/basic.html

Meanwhile, you can connect to the computers with ROS (within the school network) using ssh -x <username>@<ip_address>. Both username and IP is specified on this site, as well as the password.