C# - Basic ThreadPool Example

23. August 2011 22:07

 

The thread pool in c# is very simple to use. Though I should point out that it should only really be used to background resonable short tasks. It should not be used as an attempt to background long running tasks. Some people have tried and have failed as a result of doing these sorts of things. You should treat the thread pool as a limited resource of threads which should only ever contain a queue of short tasks.

 

If you do create long running tasks you run the risk of starving newly queued items on the thread queue. This resource starvation can be controlled by setting the min / max numbers of threads in the thread pool. Only for large applications should these ever need to be increased.

 

Here is a short example on using the thread pool.

 

 

class Program
{
	static void Main(string[] args)
	{
		ThreadPool.QueueUserWorkItem(new WaitCallback(Run), null);

		for (int i = 0; i < 5; i++)
		{
			Thread.Sleep(1000);
			Console.WriteLine("Main {0}", i);
		}

		Console.WriteLine("Main Program Finished");
		Console.ReadLine();
	}

	static void Run(object args)
	{

		for (int i = 0; i < 10; i++)
		{
			Thread.Sleep(1000);
			Console.WriteLine("Run {0}", i);
		}

	}
}

 

If you run the program output you will see that both functions appear to be running at the same time. They are actually both running at the same time on different threads. The alternative way to think of this is two programs each with their own stack and sharing the same memory space.

E-mail Kick it! DZone it! del.icio.us Permalink


Windows 7 - Explorer Thumbs.db Bug

22. August 2011 22:10

 

I finally made the move to windows 7 and this is probably the first bug I have noticed. I noticed this within 24 hours of installing the system. It would appear after 11 versions of windows microsoft still cannot get it right. All I am doing to trigger this is to enter a folder in explorer which is on a network drive. If it contains an image / video then when you come up a level and attempt to delete the folder it will first warn you that you are attempting to remove an important system file.

Like so

 

 

So you are probably thinking that this is nothing more than an annoyance. Until you press yes and you get presented with the following message.

 

 

Thats right the program that has the Thumbs.db open is actually explorer. Even though I am attempting to delete the folder using explorer. Something that I would think would be a very common operation for most people which would be to check the contents of a folder just before deleting it. So I probably cannot be the only person who has seen this really obvious bug.

 

Windows 7 may look very pritty but still seems to fail on basic file operations. I also spotted the issue on windows vista doing the same. I guess its just another thing that Microsoft *might* get around to fixing so day.

E-mail Kick it! DZone it! del.icio.us Permalink


MSSQL - Saving memory by using stored procedures

11. August 2011 08:00

 

Normally when people are trying to put a point across about using stored procedures in an application. The most common thing that is normally pointed out is about network bandwidth is normally presented. Some other things like memory saving are normally missed. This is an example on how to lower the memory footprint in sql server when using stored procedures instead of sql queries being stored in the application and being passed.

 

If you have looked into this before then you will be aware that using a stored procedure will save sql server compiling execution plans when using stored procedures. This also saves some memory. The following example will show how and just how much memory is being saved for a single query.

 

In the following example I have a service running collecting netflow packets from a cisco router and saving these in an sql table that has the following format.

 

 

CREATE TABLE [dbo].[Flow](
	[FlowID] [uniqueidentifier] NOT NULL,
	[Router] [varchar](15) NOT NULL,
	[Source] [varchar](15) NOT NULL,
	[SourcePort] [int] NOT NULL,
	[Destination] [varchar](15) NOT NULL,
	[DestinationPort] [int] NOT NULL,
	[InterfaceIn] [int] NOT NULL,
	[InterfaceOut] [int] NOT NULL,
	[Packets] [bigint] NOT NULL,
	[Bytes] [bigint] NOT NULL,
	[Protocol] [int] NOT NULL,
	[LoggedAt] [datetime] NOT NULL,
 CONSTRAINT [PK_Flow] PRIMARY KEY CLUSTERED 
(
	[FlowID] ASC
)WITH (PAD_INDEX  = OFF, STATISTICS_NORECOMPUTE  = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS  = ON, ALLOW_PAGE_LOCKS  = ON) ON [PRIMARY]
) ON [PRIMARY]
GO

ALTER TABLE [dbo].[Flow] ADD  CONSTRAINT [DF_Flow_FlowID2]  DEFAULT (newsequentialid()) FOR [FlowID]
GO

 

 

The data is being stored with the following insert statement which is being called from the service each time.

 

 

INSERT INTO Flow (Router, Source, SourcePort, Destination, DestinationPort, InterfaceIn, InterfaceOut, Packets, Bytes, Protocol, LoggedAt) VALUES(@Router, @Source, @SourcePort, @Destination, @DestinationPort, @InterfaceIn, @InterfaceOut, @Packets, @Bytes, @Protocol, @LoggedAt)

 

 

This looks stright forward enough. However if you examine the query plan cache by using the following sql.

 

SELECT
	P.usecounts AS TotalCount,
	P.size_in_bytes,
	SqlText.text
	FROM sys.dm_exec_cached_plans AS P
		CROSS APPLY sys.dm_exec_sql_text(P.plan_handle) AS SqlText
	WHERE SqlText.text like '(@Router%'
	ORDER BY P.usecounts DESC

 

 

It will show how many times the sql has been compiled to an execution plan and cached. It will also give usage counts for each of the plan caches. The following has been trimmed. There was originally about 30 rows of output for a single query.

 

 

TotalCount	size_in_bytes	(No column name)
56192	65536	(@Router nvarchar(12),@Source nvarchar(10),@Sourc
32482	65536	(@Router nvarchar(12),@Source nvarchar(11),@Sourc
15267	65536	(@Router nvarchar(12),@Source nvarchar(10),@Sourc
14742	65536	(@Router nvarchar(12),@Source nvarchar(11),@Sourc
11676	65536	(@Router nvarchar(12),@Source nvarchar(10),@Sourc
10953	65536	(@Router nvarchar(12),@Source nvarchar(10),@Sourc
10612	65536	(@Router nvarchar(12),@Source nvarchar(10),@Sourc
9512 	65536	(@Router nvarchar(12),@Source nvarchar(12),@Sourc

 

 

A number of cached plans have been created because the passed sql from the program has a number of differences. In particular as the length of the strings change the sql is using different types because the data length of the type has changed and sql server considers this to be a different type being passed. This causes sql server to create a new execution plan for every different length of string. This can result in a single query producing multiple execution plans which is using more memory. In my example above the total memory usage for all the execution plans was around 5MBytes. This may not sound much but I was only passing 3 strings which are only variable in length between 8 and 15 long.

 

When using a stored procedure the above issue with the execution plans does not occur.

 

When changing the insert statement to a simple stored procedure. 

 

 

CREATE PROCEDURE [dbo].[FlowInsert]
	@Router VARCHAR(15),
	@Source VARCHAR(15),
	@SourcePort int,
	@Destination VARCHAR(15),
	@DestinationPort int,
	@InterfaceIn int,
	@InterfaceOut int,
	@Packets bigint,
	@Bytes bigint,
	@Protocol int,
	@LoggedAt datetime
AS

INSERT INTO Flow
(Router, Source, SourcePort, Destination, DestinationPort, InterfaceIn, InterfaceOut, Packets, Bytes, Protocol, LoggedAt)
VALUES(@Router, @Source, @SourcePort, @Destination, @DestinationPort, @InterfaceIn, @InterfaceOut, @Packets, @Bytes, @Protocol, @LoggedAt)

GO

 

Then looking into the execution plan cache again using the sql above it will be reduced to a single entry using a total of 57kbytes of memory. That is saving around 4-5MBytes of memory which sql server can use else where. This may not sound like much but considering that it is a single query and it can be common for even small applications to have 100 or more quries it can be saving quite significant amounts of memory.

E-mail Kick it! DZone it! del.icio.us Permalink


C# - MD5Sum / SHASum

10. August 2011 18:00

 

In c# it can be very easy to generate a md5sum or an shasum. This can be done by using the CryptoServiceProvider which is in the System.Security.Cryptography name space.

 

The following function will generate a md5sum for a string of text

 

 

static string Md5Sum(string Input)
{
    MD5CryptoServiceProvider Crpyt = new MD5CryptoServiceProvider();

    byte[] buf = ASCIIEncoding.ASCII.GetBytes(Input);
    return BitConverter.ToString(Crpyt.ComputeHash(buf)).Replace("-", "").ToLower();
}

 

 

Or it can be changed to produce a sha checksum for a string of text in the following way.

 

 

static string ShaSum(string Input)
{
    SHA512CryptoServiceProvider Crpyt = new SHA512CryptoServiceProvider();

    byte[] buf = ASCIIEncoding.ASCII.GetBytes(Input);
    return BitConverter.ToString(Crpyt.ComputeHash(buf)).Replace("-", "").ToLower();
}

 

E-mail Kick it! DZone it! del.icio.us Permalink


MSSQL - Last Backup time / size

9. August 2011 18:00

 

In mssql server it is possible to find the last backup time as well as the largest size or compressed size of the backups. This can be down by listing the databases using the system table sys.databases and reading from the msdb.dbo.backupset which is one of the tables that is updated when a backup is taken on the server. The sizes returned will be in bytes.

 

SELECT db.name AS DBName,
	COALESCE(CONVERT(VARCHAR(MAX), MAX(bs.backup_finish_date), 101), 'N/A') as LastBackup,
	MAX(bs.backup_size) AS LargestBackup,
	MAX(bs.backup_size) AS LargestCompressedBackup
	FROM sys.databases db
	LEFT OUTER JOIN msdb.dbo.backupset bs ON bs.database_name = db.name
GROUP BY db.name
ORDER BY db.name

 

It would also be possible to include the maximum time a backup has taken for a database by modifiing the sql above and using datediff and max on the start and finish time of the backup. So the following can be used to find out how much space and time a backup will be required for each database.

 

SELECT db.name AS DBName,
	COALESCE(CONVERT(VARCHAR(MAX), MAX(bs.backup_finish_date), 101), 'N/A') as LastBackup,
	COALESCE(CONVERT(VARCHAR(MAX), MAX(DATEDIFF(second, bs.backup_start_date, bs.backup_finish_date)), 101), 'N/A') as RunTime,
	MAX(bs.backup_size) AS LargestBackup,
	MAX(bs.backup_size) AS LargestCompressedBackup
	FROM sys.databases db
	LEFT OUTER JOIN msdb.dbo.backupset bs ON bs.database_name = db.name
GROUP BY db.name
ORDER BY db.name

 

The msdb.dbo.backupset is usful for finding out the entire history of backups taken on a database. More information can be found out about it at http://technet.microsoft.com/en-us/library/ms186299.aspx

 

E-mail Kick it! DZone it! del.icio.us Permalink