I finally got my hands on a rasberry pi. So of course the first thing I did was attempt to play video. For this I turned to gstreamer and spent about 2-3 evening trying to get the omx gstreamer elements to work. Though I seemed to keep failing for various reasons. I managed to reach the stage of getting it to decode and play a single h264 frame. It would then hang and never play any further video.

So I went in search of some other examples. I found a small example app from broadcom which was actually located on the debian distribution that shipped with the pi. This was the "hello video" sample app which seemed to play the h264 file that came with it fine.

Running the sample app seemed to simple.

cd /opt/vc/src/hello_pi/hello_video
./hello_video.bin test.h264

This gets me playing. When looking into the source of the application I then notice that it was not going anything clever with the h264 eg seeks. It was in fact playing the h264 file as a stream. Therefore decided the following would also work.

cat test.h264 | ./hello_video.bin /dev/stdin

Thats great. That would mean that it would also work with nc (netcat) and the video can be sent from another machine.

On Pi

nc -l -p 10001 | ./hello_video.bin /dev/stdin

On other linux machine

gst-launch-0.10 videotestsrc ! \
 video/x-raw-yuv, width=720, height=402, framerate=30/1 ! \
 ffmpegcolorspace ! x264enc byte-stream=true ! \
 tcpclientsink host=<ip address of pi> port=10001

That managed to put up the nice gstreamer videotest source on the pi. So then I started to look into playing some real video which also involved audio. So this also turns the pi into an instant audio server.

gst-launch -v tcpserversrc host=0.0.0.0 port=10002 protocol=1 ! \
 audioconvert ! audioresample ! queue ! autoaudiosink

So by simply using the pi as a video and audio server you can play any video by using the source machine to split the audio / video and send them to the seperate service like this.

On the pi start the video / audio server

nc -l -p 10001 | ./hello_video.bin /dev/stdin

gst-launch -v tcpserversrc host=0.0.0.0 port=10002 protocol=1 ! \
audioconvert ! audioresample ! queue ! autoaudiosink

On your other linux machine run the player / re-encoder and send it to the pi

gst-launch-0.10 filesrc location=Movie.avi ! \
 decodebin2 name=dec ! queue ! ffmpegcolorspace ! \
 x264enc byte-stream=true psy-tune=1 speed-preset=1 ! \
 tcpclientsink host= port=10001 \
 dec. ! tcpclientsink host= port=10002 protocol=1

Now for the next problem. Technically the above works. Though the audio quaility is somewhat questionabile. The video quality seems to definatly pass even though in the above gstreamer command I have tunned the x264 encoder to the fastest possible setting (I have a slow machine)

Something that is definatly intresting is how the pi performs with the above setup. The audio playing seems to struggle mosty because of the driver issues and its gstreamer processes is using around 10% cpu time of the pi. The video player is split into two parts the netcat process uses around 2% cpu time and the h264 sample application also uses around 2% cpu time since it is a hardware decode. So thats a total of 14% of cpu time to play a movie.

Hopefully this opens up some cool options for some people to experement. I can already think of something useful todo with a pi, wall projector, gstreamer and ximagesrc

This is an example about why not to use the boost::shared_ptr with a new array with an explenation of why it will be dangerous to use.

Effectivly when using boost::shared_ptr or the normal std::shared_ptr when it releases the pointer it will always call delete. However when using new[] you need to use delete[]. This is the simple rule of when using malloc you use free. Then using new you use delete. When using new[] you need to use delete[]. Using a mixed combination will result at best a program crash. Or at worse siliently corrupting the heap and making a mess of your data.

The following code example which is using boost::shared_ptr correctly. Which is to allocate a new class and then the class will be automatically deleted once the last shared_ptr holding the reference has been removed.

#include <stdio.h>
#include <boost/shared_ptr.hpp>


class MyClass {
	public:
		MyClass() { 
			printf("MyClass Created\n");
		};
		
		~MyClass() {
			printf("MyClass Deleted\n");
		};
};


int main(int argc, char **argv) {
	boost::shared_ptr<MyClass> ptr;
	
	ptr.reset(new MyClass());

	return 0;
}

When running under valgrind we get the following output.

==5868== Memcheck, a memory error detector
==5868== Copyright (C) 2002-2010, and GNU GPL'd, by Julian Seward et al.
==5868== Using Valgrind-3.6.0.SVN-Debian and LibVEX; rerun with -h for copyright info
==5868== Command: ./a.out
==5868==
MyClass Created
MyClass Deleted
==5868==
==5868== HEAP SUMMARY:
==5868==     in use at exit: 0 bytes in 0 blocks
==5868==   total heap usage: 2 allocs, 2 frees, 17 bytes allocated
==5868==
==5868== All heap blocks were freed -- no leaks are possible
==5868==
==5868== For counts of detected and suppressed errors, rerun with: -v
==5868== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 19 from 8)

Once we modify the program slight to create an array of items like below. In this case ten items of "MyClass" things will change significantly.

int main(int argc, char **argv) {
	boost::shared_ptr<MyClass> ptr;
	
	ptr.reset(new MyClass[10]);

	return 0;
}

When run under valgrind this will show as a memory leak. You can see that there has been ten instances of the class created but only one instance has been deleted.

==5889== Memcheck, a memory error detector
==5889== Copyright (C) 2002-2010, and GNU GPL'd, by Julian Seward et al.
==5889== Using Valgrind-3.6.0.SVN-Debian and LibVEX; rerun with -h for copyright info
==5889== Command: ./a.out
==5889==
MyClass Created
MyClass Created
MyClass Created
MyClass Created
MyClass Created
MyClass Created
MyClass Created
MyClass Created
MyClass Created
MyClass Created
MyClass Deleted
==5889== Invalid free() / delete / delete[]
==5889==    at 0x4023881: operator delete(void*) (vg_replace_malloc.c:387)
==5889==    by 0x8048CC4: void boost::checked_delete<MyClass>(MyClass*) (in /home/james/tmp/posts/a.out)
==5889==    by 0x8048D3F: boost::detail::sp_counted_impl_p<MyClass>::dispose() (in /home/james/tmp/posts/a.out)
==5889==    by 0x80489F7: boost::detail::sp_counted_base::release() (in /home/james/tmp/posts/a.out)
==5889==    by 0x8048A69: boost::detail::shared_count::~shared_count() (in /home/james/tmp/posts/a.out)
==5889==    by 0x8048ACF: boost::shared_ptr<MyClass>::~shared_ptr() (in /home/james/tmp/posts/a.out)
==5889==    by 0x80488C8: main (in /home/james/tmp/posts/a.out)
==5889==  Address 0x42c402c is 4 bytes inside a block of size 14 alloc'd
==5889==    at 0x40243B0: operator new[](unsigned int) (vg_replace_malloc.c:299)
==5889==    by 0x8048827: main (in /home/james/tmp/posts/a.out)
==5889==
==5889==
==5889== HEAP SUMMARY:
==5889==     in use at exit: 14 bytes in 1 blocks
==5889==   total heap usage: 2 allocs, 2 frees, 30 bytes allocated
==5889==
==5889== LEAK SUMMARY:
==5889==    definitely lost: 14 bytes in 1 blocks
==5889==    indirectly lost: 0 bytes in 0 blocks
==5889==      possibly lost: 0 bytes in 0 blocks
==5889==    still reachable: 0 bytes in 0 blocks
==5889==         suppressed: 0 bytes in 0 blocks
==5889== Rerun with --leak-check=full to see details of leaked memory
==5889==
==5889== For counts of detected and suppressed errors, rerun with: -v
==5889== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 19 from 8)

However when run normally outside of valgrind using the normal c++ memory allocater in linux it will produce a heap corruption instead.

MyClass Created
MyClass Created
MyClass Created
MyClass Created
MyClass Created
MyClass Created
MyClass Created
MyClass Created
MyClass Created
MyClass Created
MyClass Deleted
*** glibc detected *** ./a.out: munmap_chunk(): invalid pointer: 0x0809e00c ***
======= Backtrace: =========
/lib/i686/cmov/libc.so.6(+0x6b381)[0xb74cc381]
/lib/i686/cmov/libc.so.6(+0x6c5fe)[0xb74cd5fe]
/usr/lib/libstdc++.so.6(_ZdlPv+0x21)[0xb76a8701]
./a.out[0x8048cc5]
./a.out[0x8048d40]
./a.out[0x80489f8]
./a.out[0x8048a6a]
./a.out[0x8048ad0]
./a.out[0x80488c9]
/lib/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xb7477ca6]
./a.out[0x8048771]
======= Memory map: ========
08048000-0804a000 r-xp 00000000 08:11 4063698    /home/james/tmp/posts/a.out
0804a000-0804b000 rw-p 00001000 08:11 4063698    /home/james/tmp/posts/a.out
0809e000-080bf000 rw-p 00000000 00:00 0          [heap]
b7460000-b7461000 rw-p 00000000 00:00 0
b7461000-b75a1000 r-xp 00000000 08:01 586488     /lib/i686/cmov/libc-2.11.3.so
b75a1000-b75a2000 ---p 00140000 08:01 586488     /lib/i686/cmov/libc-2.11.3.so
b75a2000-b75a4000 r--p 00140000 08:01 586488     /lib/i686/cmov/libc-2.11.3.so
b75a4000-b75a5000 rw-p 00142000 08:01 586488     /lib/i686/cmov/libc-2.11.3.so
b75a5000-b75a8000 rw-p 00000000 00:00 0
b75a8000-b75c5000 r-xp 00000000 08:01 578602     /lib/libgcc_s.so.1
b75c5000-b75c6000 rw-p 0001c000 08:01 578602     /lib/libgcc_s.so.1
b75c6000-b75c7000 rw-p 00000000 00:00 0

[snip]

Aborted

Note that it is also possible for this to show up in valgrind as a "Mismatched free / delete / delete[]" message as well as memory leaks. Or really anything random may happen because mixing free / delete / delete[] will result in undefined behaviour.

Correcting the issue if you have come across it is easy. Simply replace the shared_ptr with a shared_array. Of course when using shared_array you can no longer use "new MyClass()" since this will then be calling delete[] instead of delete and the same problem will occur.

Here is the correct example.

#include <stdio.h>
#include <boost/shared_array.hpp>


class MyClass {
	public:
		MyClass() { 
			printf("MyClass Created\n");
		};
		
		~MyClass() {
			printf("MyClass Deleted\n");
		};
};


int main(int argc, char **argv) {
	boost::shared_array<MyClass> ptr;
	
	ptr.reset(new MyClass[10]);

	return 0;
}

When run under valgrind you can see that it no longer leaks / crashes or complains about anything and the same number of items being created are being deleted again.

==6074== Memcheck, a memory error detector
==6074== Copyright (C) 2002-2010, and GNU GPL'd, by Julian Seward et al.
==6074== Using Valgrind-3.6.0.SVN-Debian and LibVEX; rerun with -h for copyright info
==6074== Command: ./a.out
==6074==
MyClass Created
MyClass Created
MyClass Created
MyClass Created
MyClass Created
MyClass Created
MyClass Created
MyClass Created
MyClass Created
MyClass Created
MyClass Deleted
MyClass Deleted
MyClass Deleted
MyClass Deleted
MyClass Deleted
MyClass Deleted
MyClass Deleted
MyClass Deleted
MyClass Deleted
MyClass Deleted
==6074==
==6074== HEAP SUMMARY:
==6074==     in use at exit: 0 bytes in 0 blocks
==6074==   total heap usage: 3 allocs, 3 frees, 54 bytes allocated
==6074==
==6074== All heap blocks were freed -- no leaks are possible
==6074==
==6074== For counts of detected and suppressed errors, rerun with: -v
==6074== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 19 from 8)

This is a short introduction to using the inofity api in linux to detect file system modifications for specific files. Which can be useful for a number of reasons. The particular reason why I wrote the example program here was to reload parts of apache when python scripts were being modified while doing some web development. the problem being that apache + django caches the python files.

The example here is simple. It will take a file list to monitor and will then execute a command when one of thoose files are modified. Though it also shows a simple example of how to use the inofity api in linux.

To start with you will need to get a suitable file descriptor by calling inotify_init and then setup the files you would like to monitor you can do this by using the function inotify_add_watch with the correct arguments to be notified when files changes. See below for the first half of the program. Note that the program expects the file list to be passed on the command line.

int main(int argc, char **argv) {
	int fdnotify = -1;
	int i;

	fdnotify = inotify_init();
	if (fdnotify < 0) {
		fprintf(stderr, "inotify_init failed: %s\n", strerror(errno));
		exit(-1);
	}

	for(i = 2;i<argc;i++) {
		int wd = inotify_add_watch(fdnotify, argv[i], IN_MODIFY);
		if (wd < 0) {
			fprintf(stderr, "inotify_add_watch failed: %s\n", strerror(errno));
		}
	}

Once the initial list has been completed you just need to wait for a notification which is done by reading from the file decriptor with the inotify_event data structure and just keep reading from that file descriptor in a loop.

while(1) {
		char buffer[4096];
		struct inotify_event *event = NULL;
		int exec = 0;

		int len = read(fdnotify, buffer, sizeof(buffer));
		if (len < 0) {
			fprintf(stderr, "read: %s\n", strerror(errno));
			exit(-1);
		}

		event = (struct inotify_event *) buffer;
		while(event != NULL) {
			if ( (event->mask & IN_MODIFY) && event->len > 0) {
				printf("File Modified: %s\n", event->name);
				exec = 1;
			} else {
				printf("Unknown Mask 0x%.8x\n", event->mask);
			}
				
			/* Move to next struct */	
			len -= sizeof(*event) + event->len;
			if (len > 0)
				event = ((void *) event) + sizeof(event) + event->len;
			else
				event = NULL;
		}
	
		if (exec)	
			system(argv[1]);
	}

And that is really it. Note that I wanted to "eat" most of the file notification events before running the external script so that it would only execute the script once. It can also serve other nice purposes eg it can be used to auto rebuild a c project on the file by using it like this "./inotify make `find . -type f -name '*.[ch]'" which will run make any time any of the .c, .h files change. the program has 100's of useful ways to be used for being able to trigger shell scripts from filesystem events.

Here is a copy of the full program

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>
#include <unistd.h>

#include <sys/inotify.h>

void printhelp(FILE *out, char *app) {
	fprintf(out, "Usage: %s <command> <dirs to watch>\n", app);
}

int main(int argc, char **argv) {
	int fdnotify = -1;
	int i;

	fdnotify = inotify_init();
	if (fdnotify < 0) {
		fprintf(stderr, "inotify_init failed: %s\n", strerror(errno));
		exit(-1);
	}

	if (argc < 3) {
		fprintf(stderr, argv[0]);
		exit(-1);
	}

	for(i = 2;i<argc;i++) {
		int wd = inotify_add_watch(fdnotify, argv[i], IN_MODIFY);
		if (wd < 0) {
			fprintf(stderr, "inotify_add_watch failed: %s\n", strerror(errno));
		}
	}

	while(1) {
		char buffer[4096];
		struct inotify_event *event = NULL;
		int exec = 0;

		int len = read(fdnotify, buffer, sizeof(buffer));
		if (len < 0) {
			fprintf(stderr, "read: %s\n", strerror(errno));
			exit(-1);
		}

		event = (struct inotify_event *) buffer;
		while(event != NULL) {
			if ( (event->mask & IN_MODIFY) && event->len > 0) {
				printf("File Modified: %s\n", event->name);
				exec = 1;
			} else {
				printf("Unknown Mask 0x%.8x\n", event->mask);
			}
				
			/* Move to next struct */	
			len -= sizeof(*event) + event->len;
			if (len > 0)
				event = ((void *) event) + sizeof(event) + event->len;
			else
				event = NULL;
		}
	
		if (exec)	
			system(argv[1]);
	}

	return 0;
}

In python most people are familure with a combination of if / else or a while loop. Did you know you can combine a while with an else statement. the obvious main advantage here is to prevent using extra variables and nested statement which makes the code shorter and clearer to understand.

This short example demonstrates this functionality.

lst = []
while lst:
    print "There is a list"
    break
else:
    print "Empty List"

In the case above because the contents of the while loop are no executed because the list is empty it will print "Empty List" instead. This can also work with for loops like in the following example.

for x in lst:
    if x == "Something":
        DoSomething()
        break
else:
    print "Not Found"

You should probably take notice of the break statements in the above code as the else statements runs on the last evaluation of the loop condition so when a loop terminates because its condition is false the else will always run. The break statement changes this so that the loop conditition is true and the else will be skipped.

This is a short guide on how to check if an ip address is inside a certain ip / mask combination. An example would be to ensure that an ip address like "192.168.0.10" is inside "192.168.0.0/255.255.255.0"

One of the problems that you will face is that the standard functions for converting ip addresses to an unsigned int typically will also convert the address to network byte order which will not make. You could use the ntohl function to convert it back or a simple parsing functions like below to convert it to an unsigned long using sscanf.

uint32_t IPToUInt(const std::string ip) {
    int a, b, c, d;
    uint32_t addr = 0;

    if (sscanf(ip.c_str(), "%d.%d.%d.%d", &a, &b, &c, &d) != 4)
        return 0;

    addr = a << 24;
    addr |= b << 16;
    addr |= c << 8;
    addr |= d;
    return addr;
}

The rest is the very stright forward since an ip address is really just a 32bit number if you convert the ip address the subnet and the mask a stright check can be performed like this.

bool IsIPInRange(const std::string ip, const std::string network, const std::string mask) {
    uint32_t ip_addr = IPToUInt(ip);
    uint32_t network_addr = IPToUInt(network);
    uint32_t mask_addr = IPToUInt(mask);

    uint32_t net_lower = (network_addr & mask_addr);
    uint32_t net_upper = (net_lower | (~mask_addr));

    if (ip_addr >= net_lower &&
        ip_addr <= net_upper)
        return true;
    return false;
}

And just to make sure that it works here is some test code.

void test(const std::string ip, const std::string network, const std::string mask, bool expected) {
    if (IsIPInRange(ip, network, mask) != expected) {
        printf("Failed! %s %s %s %s\n", ip.c_str(), network.c_str(), mask.c_str(), expected ? "True" : "False");
    } else {
        printf("Success! %s %s %s %s\n", ip.c_str(), network.c_str(), mask.c_str(), expected ? "True" : "False");
    }
}


int main(int argc, char **argv) {
    std::string ip(argv[1]);

    test("192.168.1.1", "192.168.1.0", "255.255.255.0", true);
    test("192.168.1.1", "192.168.1.2", "255.255.255.255", false);
    test("192.168.1.3", "192.168.1.2", "255.255.255.255", false);

    test("220.1.1.22", "192.168.1.0", "255.255.255.0", false);
    test("220.1.1.22", "220.1.1.22", "255.255.255.255", true);
    test("220.1.1.22", "220.1.1.23", "255.255.255.255", false);
    test("220.1.1.22", "220.1.1.21", "255.255.255.255", false);

    test("0.0.0.1", "0.0.0.0", "0.0.0.0", true);
    test("192.168.1.2", "10.0.0.1", "255.255.255.255", false);

    return 0;
}