Doing for loops in bash

9. May 2012 23:21

 

It can be commonly known that you can do for loops in bash by doing something like for i in * ; do echo $i ; done or some such. This will loop for every file in the current directory. Though typical for loops for a specific number range can be a little more difficult since you need to form the data to be able to execute the loop.

 

To get a simple for loop to work we can copy python's for i in range(x, y): type of loop since the bash for loop is exactly the same as this. Both bash / python perform a for each loop around a list of data items rather than the traditional for loop with a counter. This can be used to our advantage since all you need to do is create a small program that generates this data lists.

 

The following c program does this.

 

#include <stdio.h>
#include <stdlib.h>

void print_usage(FILE *fp, char *app) {
    fprintf(fp, "Usage: %s <start number> <end number>\n", app);
    fprintf(fp, "\n");
}

int main(int argc, char **argv) {
    int a = 0, b = 0;
    int i;

    if (argc < 3) {
        print_usage(stderr, argv[0]);
        exit(EXIT_FAILURE);
    }

    a = atoi(argv[1]);
    b = atoi(argv[2]);
    
    if (a >= b) {
		int tmp = a;
		b = a;
		a = tmp;
    }

    for(i=a;i<=b;i++) {
        printf("%d\n", i);
    }

    return 0;
}

All you need to do is put the program above into a c file and compile it with gcc (gcc -Wall range.c -o range) and place the executable on the path (eg in $HOME/bin). Then you can do for loops in the bash shell the following way.

 

for i in `range 0 20` ; do echo $i ; done

The above will of course produce the output of 0 to 20 on the terminal when run.

 

E-mail Kick it! DZone it! del.icio.us Permalink


C/C++ - Generate a random number

19. April 2012 21:57

 

This is a short example for generating a random number in C/C++. It is actually easy to do. In this case we can produce a random number between 0 and 10 and print it out.

#include <stdio.h>
#include <stdlib.h>
#include <time.h>


int main(int argc, char **argv) {

	srand(time(NULL));

	for(int i =0;i<10;i++) {
		int r = rand() % 10;
		printf("%d\n", r);
	}

	return 0;
}

 

The solution above will only work for generating a number from 0 to RAND_MAX which is typically 32768.

E-mail Kick it! DZone it! del.icio.us Permalink


Floating point comparisons don't work. Don't event attempt them

12. April 2012 22:23

 

This started because somebody discovered and issue with php. Which turns out to also pop up in other languages like php, javascript and of course python. These magic numbers happen to be an edge case for a double precision floating point number. So it actually happens in all languages. Simply put the number is large enough to start dropping the least significant digits.

 

One of these numbers happen to be 9223372036854775807.0

 

Here is an example of what the problem is.

 

>>> 9223372036854775807 == 9223372036854775808
False
>>> 9223372036854775807.0 == 9223372036854775808
True

 

Obviously you would think that the 2nd should also equal false. However in a floating point number they are actually converted to the same number. So of course they actually appear to be equal. We can show this by doing the following.

 

>>> a = 9223372036854775807.0
>>> b = 9223372036854775808.0
>>> a == b
True
>>> print a
9.22337203685e+18
>>> print b
9.22337203685e+18

 

As you can see the numbers are actually the same. However when you try to compare them some other ways they also break when trying to compare when forcing the type to an int like this.

 

 

>>> int(9223372036854775807) == int(9223372036854775808)
False
>>> int(9223372036854775807.0) == int(9223372036854775808)
True
>>> int(9223372036854775807.0) == 9223372036854775808
True
>>> 9223372036854775807.0 == int(9223372036854775808)
True

Like this

 

 

>>> print int(9223372036854775807)
9223372036854775807
>>> print int(9223372036854775807.0)
9223372036854775808

 

 

However this particular problem does not apply to python alone it does actually apply to anything that is using the standard ieee 64 bit floating point since it is actually impossible to represent the number 9223372036854775807.0 to it gets rounded to the nearest floating point number that happens to be 9223372036854775808

 

We can prove this because it's also acts this way in a C compiler.

 

 

#include <stdio.h>

int main(int argc, char **argv) {
        double a = 9223372036854775807.0;
        double b = 9223372036854775808.0;

        if (a == b)
                printf("True\n");
        else
                printf("False\n");

        return 0;
}

 

And if you take it down to assembler it will also show that it is happening there. However if you look at the raw data in the exe file you will also see that it has actually already truncated the number 9223372036854775807.0 to the same as the other number 9223372036854775808.0

 

Just to make it stick a little more the following is exactly the same issue!

 

>>> a = 9999999999999999.00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001
>>> b = 9999999999999999.00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000002
>>> a == b
True

 

Its floating point. Don't attempt to compare them ever to be equal to each other! It doesn't work with large numbers because there is not enough accuracy to store the information.

E-mail Kick it! DZone it! del.icio.us Permalink


Using gdb to debug a core file

5. April 2012 08:00

 

The gnu debugger (gdb) is probably the best tool for looking into core files. It also isn't overly complex to use to get some basic starting information. So this is a quick guide to getting some debug information eg variable's and stack traces from a core dump which is formed when an application crashes in linux.

 

If an application crashes and doesn't produce a core file it is probably because of the limit settings you can check and enable core dumps by using the following "ulimit -c" if it outputs a 0 it will not produce a core. You can use "ulimit -c unlimited" to make the core dump file size unlimited. Be aware though that if you have a lot of crashes it can use a significant amount of disk space.

 

Let start with finding out what made a core file in the first place. In order to debug it at all you need to know exactly what program crashed. You can determin this using the following command.

 

cat core |strings |grep -E '^_='
_=./willcore.exe

 

In this case the "./willcore.exe" made the core dump. Another way to find out what core'd is to use gdb. The example is below

 

gdb --core core

Core was generated by `./willcore.exe'.
Program terminated with signal 11, Segmentation fault.
#0  0x080483d4 in ?? ()
(gdb)

 

Something to notice at this point is that is shows where the last execution point was. In this case it was running at memory location 0x080483d4. However since there are no debugging symbols loaded in gdb yet it shows ?? because it cannot translate the raw address to a function.

 

You can get gdb to load the executable and debugging symbols (assuming they are compiled into the executable) using the "file" command. I have also added the "bt" command to produce a back trace so show the execution stack.

 

 

(gdb) file ./willcore.exe
Reading symbols from /home/james/CVS-Root/linux/misc/willcore.exe...done.
(gdb) bt
#0  0x080483d4 in main (argc=1, argv=0xbf877ef4) at willcore.c:8

 

In the example above the symbols now resolve and will show a lot more information. If they do not show after loading the executable into gdb it will be a problem with debugging symbols. You will need to go and compile the program with debugging switched on eg the "-ggdb" flag in gcc and g++.

 

Now that things are loaded you can move between stack frame's using "frame <number>" where number is the part beside the # in the stack output and you can also print and inspect other parts of memory as well as list the source code from the program assuming the source still exists in the original location that it was compiled from.

 

 

(gdb) p argc
$1 = 1
(gdb) p argv[0]
$2 = 0xbf87988c "./willcore.exe"
(gdb) p argv[1]
$3 = 0x0
(gdb) list
1
2
3       #include <stdio.h>
4
5       int main(int argc, char **argv) {
6               char *tmp = 0;
7
8               *tmp = '0';
9
10
(gdb) p tmp
$4 = 0x0

 

Note: the above example shows how to produce a core file by dereferencing a null pointer in c which in its self can be useful to fore a core dump which is what I used for this tutorial.

 

As a final example the following will dump all the stacks from all threads that were running in a process at the time it crashed. Though in the sample program there only is one thread. In a large application this could provide several pages of output.

 

 

(gdb) thread apply all bt

Thread 1 (Thread 12519):
#0  0x080483d4 in main (argc=1, argv=0xbf877ef4) at willcore.c:8
(gdb)

 

E-mail Kick it! DZone it! del.icio.us Permalink


What is a core file?

4. April 2012 22:16

 

If you have an application that crashes with a "segment fault" or other error error message like SIGBUS or SIGILL. There are an indication that a serious problem has occured inside the application that was running. eg a segment fault means that the application tried to access memory that is not allocated to that process. Or a sig ill is an illegal instruction and this occurs when a program jumps into random memory and the processor cannot understand the instruction at the current address that it is running at.

 

If the program didn't actually dump a core there can be a number of reasons for this. Typically because the ulimit value is set to 0. You can check this by running the command "ulimit -c" and it will show you the maximum size a core dump can be. You can set this to unliminted by using the command "ulimit -c unlimited" this will typically cause a core to be dumped. There are some other security releated reason why a core will not be dumped. An example is if an executable is marked suid it won't dump a core by default.

 

Many people always wonder what a core file is and a really simple way to explain it is that it contains the core memory of the application that crashed. Though it is in a format simalar the elf executable file format. This provides information about memory region and their contents. You can show the contents of a core file using the objdump utility.

 

eg the output of the command "objdump -x core" will show something like this.

 

 

core:     file format elf32-i386
core
architecture: i386, flags 0x00000000:

start address 0x00000000

Program Header:
    NOTE off    0x00000294 vaddr 0x00000000 paddr 0x00000000 align 2**0
         filesz 0x0000022c memsz 0x00000000 flags ---
    LOAD off    0x00001000 vaddr 0x08048000 paddr 0x00000000 align 2**12
         filesz 0x00001000 memsz 0x00001000 flags r-x
    LOAD off    0x00002000 vaddr 0x08049000 paddr 0x00000000 align 2**12
         filesz 0x00001000 memsz 0x00001000 flags rw-
    LOAD off    0x00003000 vaddr 0xb75f9000 paddr 0x00000000 align 2**12
         filesz 0x00002000 memsz 0x00002000 flags rw-
    LOAD off    0x00005000 vaddr 0xb75fb000 paddr 0x00000000 align 2**12
         filesz 0x00001000 memsz 0x00140000 flags r-x
    LOAD off    0x00006000 vaddr 0xb773b000 paddr 0x00000000 align 2**12
         filesz 0x00000000 memsz 0x00001000 flags ---
    LOAD off    0x00006000 vaddr 0xb773c000 paddr 0x00000000 align 2**12
         filesz 0x00002000 memsz 0x00002000 flags r--
    LOAD off    0x00008000 vaddr 0xb773e000 paddr 0x00000000 align 2**12
         filesz 0x00001000 memsz 0x00001000 flags rw-
    LOAD off    0x00009000 vaddr 0xb773f000 paddr 0x00000000 align 2**12
         filesz 0x00003000 memsz 0x00003000 flags rw-
    LOAD off    0x0000c000 vaddr 0xb7742000 paddr 0x00000000 align 2**12
         filesz 0x00001000 memsz 0x00015000 flags r-x
    LOAD off    0x0000d000 vaddr 0xb7757000 paddr 0x00000000 align 2**12
         filesz 0x00001000 memsz 0x00001000 flags r--
    LOAD off    0x0000e000 vaddr 0xb7758000 paddr 0x00000000 align 2**12
         filesz 0x00001000 memsz 0x00001000 flags rw-
    LOAD off    0x0000f000 vaddr 0xb7759000 paddr 0x00000000 align 2**12
         filesz 0x00002000 memsz 0x00002000 flags rw-
    LOAD off    0x00011000 vaddr 0xb7774000 paddr 0x00000000 align 2**12
         filesz 0x00002000 memsz 0x00002000 flags rw-
    LOAD off    0x00013000 vaddr 0xb7776000 paddr 0x00000000 align 2**12
         filesz 0x00001000 memsz 0x00001000 flags r-x
    LOAD off    0x00014000 vaddr 0xb7777000 paddr 0x00000000 align 2**12
         filesz 0x00001000 memsz 0x0001b000 flags r-x
    LOAD off    0x00015000 vaddr 0xb7792000 paddr 0x00000000 align 2**12
         filesz 0x00001000 memsz 0x00001000 flags r--
    LOAD off    0x00016000 vaddr 0xb7793000 paddr 0x00000000 align 2**12
         filesz 0x00001000 memsz 0x00001000 flags rw-
    LOAD off    0x00017000 vaddr 0xbfe74000 paddr 0x00000000 align 2**12
         filesz 0x00016000 memsz 0x00016000 flags rw-

Sections:
Idx Name          Size      VMA       LMA       File off  Algn
  0 note0         0000022c  00000000  00000000  00000294  2**0
                  CONTENTS, READONLY
  1 .reg/12315    00000044  00000000  00000000  000002f0  2**2
                  CONTENTS
  2 .reg          00000044  00000000  00000000  000002f0  2**2
                  CONTENTS
  3 .auxv         000000a0  00000000  00000000  000003dc  2**2
                  CONTENTS
  4 load1         00001000  08048000  00000000  00001000  2**12
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
  5 load2         00001000  08049000  00000000  00002000  2**12
                  CONTENTS, ALLOC, LOAD
  6 load3         00002000  b75f9000  00000000  00003000  2**12
                  CONTENTS, ALLOC, LOAD
  7 load4a        00001000  b75fb000  00000000  00005000  2**12
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
  8 load4b        00000000  b75fc000  00001000  00006000  2**12
                  ALLOC, READONLY, CODE
  9 load5         00000000  b773b000  00000000  00006000  2**12
                  ALLOC, READONLY
 10 load6         00002000  b773c000  00000000  00006000  2**12
                  CONTENTS, ALLOC, LOAD, READONLY
 11 load7         00001000  b773e000  00000000  00008000  2**12
                  CONTENTS, ALLOC, LOAD
 12 load8         00003000  b773f000  00000000  00009000  2**12
                  CONTENTS, ALLOC, LOAD
 13 load9a        00001000  b7742000  00000000  0000c000  2**12
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
 14 load9b        00000000  b7743000  00001000  0000d000  2**12
                  ALLOC, READONLY, CODE
 15 load10        00001000  b7757000  00000000  0000d000  2**12
                  CONTENTS, ALLOC, LOAD, READONLY
 16 load11        00001000  b7758000  00000000  0000e000  2**12
                  CONTENTS, ALLOC, LOAD
 17 load12        00002000  b7759000  00000000  0000f000  2**12
                  CONTENTS, ALLOC, LOAD
 18 load13        00002000  b7774000  00000000  00011000  2**12
                  CONTENTS, ALLOC, LOAD
 19 load14        00001000  b7776000  00000000  00013000  2**12
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
 20 load15a       00001000  b7777000  00000000  00014000  2**12
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
 21 load15b       00000000  b7778000  00001000  00015000  2**12
                  ALLOC, READONLY, CODE
 22 load16        00001000  b7792000  00000000  00015000  2**12
                  CONTENTS, ALLOC, LOAD, READONLY
 23 load17        00001000  b7793000  00000000  00016000  2**12
                  CONTENTS, ALLOC, LOAD
 24 load18        00016000  bfe74000  00000000  00017000  2**12
                  CONTENTS, ALLOC, LOAD
SYMBOL TABLE:
no symbols

 

 

The above information isn't really that useful but does provide a memory map with the permissions of the different regions of the application when it crashed. This can become useful later when debugging things. Like in C/C++ when things are declared "const" they are loaded into a read-only section of memory so any attempt to write to that area will fail and cause a "segment fault"

 

Something that is more usful is with the following command you can determine which application crashed by simply reading all the string for the executable and looking for the '_=' string.

 

cat core |strings |grep -E '^_='
_=./willcore.exe

 

You may also be interested in my next post which uses gdb to read a core file

E-mail Kick it! DZone it! del.icio.us Permalink