Reading from text files
Reading from text files. For this, I do not need fancy serialization of curves in parametric form. This can come later. All I need is the ability to read from data stored in plain text format, one row per time instant, each row corresponding to a vector of predetermined length and possibly its derivatives, each element being separated by spaces. So, for instance, the CoM is a three-element vector, and it should stored together with its velocity and acceleration like this:
1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0
1.1 2.1 3.1 4.1 5.1 6.1 7.1 8.1 9.1
where [1.0 2.0 3.0] is the position, [4.0 5.0 6.0] is the velocity, and [7.0 8.0 9.0] is the acceleration at instant 0, [1.1 2.1 3.1], [4.1 5.1 6.1] and [7.1 8.1 9.1] are position, velocity and acceleration at instant 1, ecc. No time specification is needed in the file, it is assumed that the object reading it knows the frequency. When the file is over, the trajectory should steadily keep issuing its final value.
Notice there are several issues in the current parametric-curves implementation in TextFile, which are the main reason why I am pushing for change. Therefore, I would really appreciate to see this issues addressed in curves, if they are not already solved. Here I list them, from most to least important (in my opinion). Unfortunately, I think the most important ones are also the most difficult to address, so good work!
- Size. This is possibly the most limiting one. Currently,
TextFilereads the file by storing its values into a buffer, but the size of this buffer is limited to 1,000,000
double. If the file is bigger that that, an error message is issued and unwanted behavior is produced (I think an empty matrix is returned). This is severely limiting and should be addressed. This might be trickier than it seems. A buffer is certainly necessary, because we cannot afford to do a disk access each time we need a new value, i.e. at each time instant. But simply increasing the buffer will just push the limit forward, not remove it. On the other hand we should avoid having huge buffers in order not to fill the memory. Maybe some work with threads will be needed
- File loading. Strictly related to the previous point. The current file loading process is blocking and can generate failure on the robot with RT constrains. We could use a separate thread to read the data. See this issue comment
- The current implementation expects the file to contain position, velocity and acceleration. If velocity and acceleration are not needed, one is obliged to provide them anyways, filling the file with dummy values, thereby greatly increasing the size of the file for nothing. It should be possible to specify we only care about values up to a certain time derivative. This might help alleviate the first point, because less values will be stored. What happens if, for instance, velocity is asked but the file only contains derivatives, do as you want, it can cause a crash, or return zero, or again try a numerical differentiation, I don't care
- The current implementation is buggy when it comes to the final instant: the buffer is read out of bounds and it returns garbage. In sot-talos-balance, I had to implement the workaround of asking the time instant just before the last one in order to avoid this bug
- The current implementation expects the position, velocity and acceleration vector to be all of the same size. This means that, for instance, we cannot include a quaternion for the orientation followed by a 3D vector for the angular velocity: we are stuck to Euler angles. It would be nice to allow for different sizes, at least between position and higher derivatives
- The current implementation seems not to be robust to slight variations of the format. For instance, if a line ends with a trailing white space, it will crash. Some flexibility will probably make out life easier