Bash - Upgrading lots of git repos.

16. August 2013 19:33

 

Here is a nice example of how to execute a git command on multiple git repositories one after another. Assuming that they are all in the same directory.

 

#!/bin/bash

for i in * ; do
	if [ -d $i ] ; then
		if [ -d $i/.git ] ; then
			echo Updating $i ....
			(cd $i ; git pull)
			echo
		fi
	fi
done

E-mail Kick it! DZone it! del.icio.us Permalink


Linux - What and how to kill a zombie process

1. March 2013 10:12

 

What is a zombie?

 

A zombie process is simply a process that has exited but its parent has not yet read the processes exit code. Since the process has exited it does not have any open files or uses any memory in fact the only resources it is using is a entry in the task list. So in most Linux systems will mean that it is using around 4k-8k of memory.

 

Until the parent process reads the exit code the zombie will exist. Typically this points to a software bug in the parent process or an issue that caused the child (which is now the zombie) to exit unexpectedly. Which would also point to a bug or issue in the program. In either case the parent process should have actually dealt with the error an acted on it to read the exit status of the process which will clean up the zombie process and allow it to exit.

 

Why is there any issue?

 

Since the program isn't really using any resources it really doesn't cause any issues. However if you have a lot of zombie's being generated inside a system due to a bug in the software it will cause an issue because it is still running as a process and sooner or later you will hit the limit of the maximum number of processes permitted to run in the system. If this limit is reached you will not be able to start any more processes. This means you won't be able to run commands from the command line any more.

 

Why does kill now work on the zombie process?

 

Kill simply does not work on a zombie process since it is in fact already "dead"

 

How to kill it?

 

Since the issue really exists in the parent process the solution is to remove the parent process either by killing it or shutting it down. This will cause the child "zombie" processes to get a new parent process which will be the parent of the parent.

 

If you are working in a complex system you may need to kill several parent process in order to eliminate the zombies until the zombies get a parent process that will read the exit code from the processes which will cause the zombie process to be remove.

 

 

Hint: if you use the command "ps axfu" the "f" in the argument list will print the output as a tree which will make finding the parent somewhat simpler.

E-mail Kick it! DZone it! del.icio.us Permalink


Debian - Getting sshfs to work

14. February 2013 20:23

 

This is a short guide to get ssh to work on debian. So this should also work with some other linux varients like ubuntu. It can be a very useful tool to be able to access your home directory from a 2nd linux machine or from your raspberry pi. I have been using it so that you can run your normal powerful code editors which the raspberry pi does not have enough resources to run them fast enough.

 

 

To Install / Setup

 

sudo apt-get install sshfs 

sudo addgroup $USER fuse

 

Note: You don't need to add your self to the fuse ground. But it is required if you want to be able to mount / umount the filesystem without root permissions.

 

 

To Use

 

mkdir Raspberry

sshfs <username>@<ip address>:/home/<username>/Raspberry  -o uid=1001,gid=1002 Raspberry

 

 

Note: in the above you should change the uid and gid to your user id and group id. You can look thoose up in the /etc/passwd and /etc/group files on the machine that you are mounting the remote filesystem on.

 

 

To unmount

 

fusermount -u /home/james/Raspberry/

 

 

 

Another useful tip is that it will also work with ssh key authentication so that you do not even require a password or so that you can have it configured to automount on boot using the crontab.

E-mail Kick it! DZone it! del.icio.us Permalink


Linux Programming - Using inotify for detecting file modifications

22. September 2012 09:21

 

This is a short introduction to using the inofity api in linux to detect file system modifications for specific files. Which can be useful for a number of reasons. The particular reason why I wrote the example program here was to reload parts of apache when python scripts were being modified while doing some web development. the problem being that apache + django caches the python files.

 

The example here is simple. It will take a file list to monitor and will then execute a command when one of thoose files are modified. Though it also shows a simple example of how to use the inofity api in linux.

 

To start with you will need to get a suitable file descriptor by calling inotify_init and then setup the files you would like to monitor you can do this by using the function inotify_add_watch with the correct arguments to be notified when files changes. See below for the first half of the program. Note that the program expects the file list to be passed on the command line.

 

int main(int argc, char **argv) {
	int fdnotify = -1;
	int i;

	fdnotify = inotify_init();
	if (fdnotify < 0) {
		fprintf(stderr, "inotify_init failed: %s\n", strerror(errno));
		exit(-1);
	}

	for(i = 2;i<argc;i++) {
		int wd = inotify_add_watch(fdnotify, argv[i], IN_MODIFY);
		if (wd < 0) {
			fprintf(stderr, "inotify_add_watch failed: %s\n", strerror(errno));
		}
	}

 

Once the initial list has been completed you just need to wait for a notification which is done by reading from the file decriptor with the inotify_event data structure and just keep reading from that file descriptor in a loop.

 

while(1) {
		char buffer[4096];
		struct inotify_event *event = NULL;
		int exec = 0;

		int len = read(fdnotify, buffer, sizeof(buffer));
		if (len < 0) {
			fprintf(stderr, "read: %s\n", strerror(errno));
			exit(-1);
		}

		event = (struct inotify_event *) buffer;
		while(event != NULL) {
			if ( (event->mask & IN_MODIFY) && event->len > 0) {
				printf("File Modified: %s\n", event->name);
				exec = 1;
			} else {
				printf("Unknown Mask 0x%.8x\n", event->mask);
			}
				
			/* Move to next struct */	
			len -= sizeof(*event) + event->len;
			if (len > 0)
				event = ((void *) event) + sizeof(event) + event->len;
			else
				event = NULL;
		}
	
		if (exec)	
			system(argv[1]);
	}

And that is really it. Note that I wanted to "eat" most of the file notification events before running the external script so that it would only execute the script once. It can also serve other nice purposes eg it can be used to auto rebuild a c project on the file by using it like this "./inotify make `find . -type f -name '*.[ch]'" which will run make any time any of the .c, .h files change. the program has 100's of useful ways to be used for being able to trigger shell scripts from filesystem events.

 

Here is a copy of the full program

 

 

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>
#include <unistd.h>

#include <sys/inotify.h>

void printhelp(FILE *out, char *app) {
	fprintf(out, "Usage: %s <command> <dirs to watch>\n", app);
}

int main(int argc, char **argv) {
	int fdnotify = -1;
	int i;

	fdnotify = inotify_init();
	if (fdnotify < 0) {
		fprintf(stderr, "inotify_init failed: %s\n", strerror(errno));
		exit(-1);
	}

	if (argc < 3) {
		fprintf(stderr, argv[0]);
		exit(-1);
	}

	for(i = 2;i<argc;i++) {
		int wd = inotify_add_watch(fdnotify, argv[i], IN_MODIFY);
		if (wd < 0) {
			fprintf(stderr, "inotify_add_watch failed: %s\n", strerror(errno));
		}
	}

	while(1) {
		char buffer[4096];
		struct inotify_event *event = NULL;
		int exec = 0;

		int len = read(fdnotify, buffer, sizeof(buffer));
		if (len < 0) {
			fprintf(stderr, "read: %s\n", strerror(errno));
			exit(-1);
		}

		event = (struct inotify_event *) buffer;
		while(event != NULL) {
			if ( (event->mask & IN_MODIFY) && event->len > 0) {
				printf("File Modified: %s\n", event->name);
				exec = 1;
			} else {
				printf("Unknown Mask 0x%.8x\n", event->mask);
			}
				
			/* Move to next struct */	
			len -= sizeof(*event) + event->len;
			if (len > 0)
				event = ((void *) event) + sizeof(event) + event->len;
			else
				event = NULL;
		}
	
		if (exec)	
			system(argv[1]);
	}

	return 0;
}

E-mail Kick it! DZone it! del.icio.us Permalink


Linux - Killing all processes for a specific user

13. July 2012 08:00

 

Here is a few methods for killing tasks for a specific user in linux which may be required during account deletion or because somebody has managed to be an idiot and locked himself out with a fork bomb or some such.

 

The simple method is to use the utility called 'slay' which for debian / unbuntu and most other distrobutions is avilable if its not install you can install it using 'apt-get install slay'

 

It is very simple to use. Just running the command slay <username> and it will kill all of that users processes.

 

 

The other method to use when slay is not avilable is a combination of ps and kill. You can use the following command

 

 

kill -TERM `ps h --User nobody -o pid`

 

 

Understanding the above can other advantages as well because you can switch out the --User for --Group and kill processes by group id as well as for a specific user.

E-mail Kick it! DZone it! del.icio.us Permalink