A good feature of InfoPath 2007

Last night, I attended MS meeting of "Overview of new Portal features in Microsoft Office SharePoint Server 2007". Although the workflow feature for document management sounds good to me, I have deep impression about a good feature of InfoPath 2007: One form everywhere!



Basically, you can design a form to deploy only once. If a user has InfoPath installed on his/her machine, the user will download and fill the form locally (smart-client version); if the user does not have InfoPath installed, he/she can use popular web browser to fill the form (browser version, slightly different from smart-client version); the user can even see the form on a mobile device.

"One form" makes development and business process much easier.

Something big - Windows Workflow Foundation (WF) and ...


"Something big is about to happen ..." is the very first statement of the excellent book "Essential Windows Workflow Foundation".

The book convinced me that WF will change our programming world by enabling domain specific language: Programmers can build domain specific WF Activities, then business people can build system using those WF Activities directly! -- That will be a really BIG thing!

Another BIG thing I realize will be: big monitor! Our current monitor is not designed to show workflow. Take a brief look at a workflow diagram, you will know my meaning. :)

SOA Messaging or bridge table?

Nowadays, when we integrate different systems together inside a company, Service Oriented Architecture (SOA) is the first choice.

But is it a really good choice? SOA is based on messaging to transfer request and response. Messaging gives us decoupling advantage, but also unreliability.

Let's suppose one case: System A sends message to System B for data update. What will happen if the message is lost for some reason (out of buffer, system failure, etc.)? Should we build shake-hand protocol between A and B to make sure the message is successfully delivered?

How about the old way -- to use a bridge table in a shared database? System A can add a new record into the bridge table for data update; System B can query the bridge table periodically (or to use database notificatione, like SQL Server 2005) to get the update. In this way, we can guarantee the update "message" is delivered from A to B.

If you think there is delay for this solution, SOA messaging also has delay. We can adjust query period for performance issues.

I am not saying SOA is bad. I just want to say when you integrate systems, old way may be a better choice than SOA.

How to break a connection in LinkedIn

I invited several friends to my linkedin account. I will invite more later, because it is an easy way to find old friends to keep in touch.

But it is hard to delete a person from my connection list: I did not find any button to break a connection. (This sounds like I hate somebody, no, I only do not want to keep many names that I can not remember well.)

Finally, I got this link somewhere:

http://www.linkedin.com/connections?displayBreakConnections=

Hopefully, when I delete a person from my connections, he/she would not receive an email like "Jun hates you! He does not want to see you any more!" :)

Column Data Encryption in SQL 2005

How to encrypt Social Security Number or Credit Card Number in SQL database? We can not use one-way hash, which can be used to verify password in some solutions. For SSN, we should be able to decrypt the data when needed.

To speed up process, we should use symmetric key – same key for encryption and decryption. When we add a new record, the key is used to encrypt the data before the record is physically added into data table; when we get the record, the same key is used to decrypt the data before it is returned to client program.

Here are the questions:

1. Where do you save the symmetric key? It is not a good idea to include plain text key in every related SQL statements or stored procedures – it is too easy to hack and too complex to change key!

Ok, I can put the symmetric key (key1) into a central table in SQL server database. To avoid hacking, I also encrypt key1 using another symmetric key (key2) – good :) But where should I put key2 then? The problem to save key2 is very similar to the initial problem to save key1.

2. When the encryption key is changed for security reason, we should decrypt the old data using the old key and then to encrypt all data using the new encryption key. Do we need track which column in which table is encrypted using which key? What will happen if somehow we forget to keep the tracking up-to-date?

Fortunately, SQL Server 2005 builds the encryption feature for us. :)

For question #1, SQL Server uses multiple levels of keys:

Symmetric key --encrypted by--> Database Master Key or certificate --encrypted by--> SQL Server Service Master Key --encrypted by--> Windows DPAPI and the service account credential or the machine key.

The benefit of multiple level keys is that you do not need to provide any password to decrypt a key. SQL Server can decrypt the key by itself. What you need is to use symmetric key name in your code.

Below is a sample from http://msdn2.microsoft.com/en-us/library/ms179331.aspx:

USE AdventureWorks;
GO

--If there is no master key, create one now.
IF NOT EXISTS
(SELECT * FROM sys.symmetric_keys WHERE symmetric_key_id = 101)
CREATE MASTER KEY ENCRYPTION BY
PASSWORD = '23987hxJKL969#ghf0%94467GRkjg5k3fd117r$$#1946kcj$n44nhdlj'
GO

CREATE CERTIFICATE HumanResources037
WITH SUBJECT = 'Employee Social Security Numbers';
GO

CREATE SYMMETRIC KEY SSN_Key_01
WITH ALGORITHM = AES_256
ENCRYPTION BY CERTIFICATE HumanResources037;
GO

USE [AdventureWorks];
GO

-- Create a column in which to store the encrypted data.
ALTER TABLE HumanResources.Employee
ADD EncryptedNationalIDNumber varbinary(128);
GO

-- Open the symmetric key with which to encrypt the data.
OPEN SYMMETRIC KEY SSN_Key_01
DECRYPTION BY CERTIFICATE HumanResources037;

-- Encrypt the value in column NationalIDNumber with symmetric
-- key SSN_Key_01. Save the result in column EncryptedNationalIDNumber.
UPDATE HumanResources.Employee
SET EncryptedNationalIDNumber = EncryptByKey(Key_GUID('SSN_Key_01'), NationalIDNumber);
GO

-- Verify the encryption.
-- First, open the symmetric key with which to decrypt the data.
OPEN SYMMETRIC KEY SSN_Key_01
DECRYPTION BY CERTIFICATE HumanResources037;
GO

-- Now list the original ID, the encrypted ID, and the
-- decrypted ciphertext. If the decryption worked, the original
-- and the decrypted ID will match.
SELECT NationalIDNumber, EncryptedNationalIDNumber
AS 'Encrypted ID Number',
CONVERT(nvarchar, DecryptByKey(EncryptedNationalIDNumber))
AS 'Decrypted ID Number'
FROM HumanResources.Employee;
GO

To support server cluster, you can export Service Master Key, Database Master Key, symmetric key and certificate to another server.

For question #2, when a key is to be changed, you should use SQL command to change or recreate a key, so that SQL Server can decrypt old data and encrypt data using the new key automatically.

The encrypted data begins with symmetric key GUID. You can define several symmetric keys to encrypt different data columns. SQL Server can get proper key using the GUID part.

Using iText to build PDF on the fly

Although there are many solutions (such as XML-FOP) to generate PDF on the fly, iText is a good choice to read PDF template and bind data dynamically.

1) For PDF form where data fields have fixed length, it is quite easy to use iText (I use C# version iTextSharp here) to bind data to PDF Text Field:

// Read PDF template
iTextSharp.text.pdf.PdfReader pdfRd = new iTextSharp.text.pdf.PdfReader(Server.MapPath("~/SampleJS.pdf"));
iTextSharp.text.pdf.PdfStamper stamp = new iTextSharp.text.pdf.PdfStamper(pdfRd, outputStream);

// Set form Text Fields
iTextSharp.text.pdf.AcroFields fields = stamp.AcroFields;
fields.SetField("formText", value);

// Close stamp
stamp.Close();

2) For free text data, you cannot put PDF Text Field to show data. For example, if you want to bind data to this template:

"Welcome to «Institution»! Please send mail to «Person» before «Date»"

You cannot use PDF Text Field because Text Field has fixed length. If «Institution» value is short, there will be extra blank space in the statement.

So what can you do? You can put the template statement into one Text Field with “Read Only” permission. Then you can replace the string with data like below:

// Get Text Fields from PDF file
iTextSharp.text.pdf.AcroFields fields = stamp.AcroFields;

// Get template
string formText = fields.GetField("formText");

// Replace template with data
formText = formText.Replace("«Institution»", txtInstitution);
formText = formText.Replace("«Person»", txtPerson);
formText = formText.Replace("«Date»", date);

// Set back the Text Field
fields.SetField("formText", formText);

3) How to insert image? PDF Text Field can be a placeholder for image. For example, to add a logo image to a placeholder:

// Get content to make changes
PdfContentByte overContent = stamp.GetOverContent(1);

// Get logo image
iTextSharp.text.Image logo = iTextSharp.text.Image.GetInstance(Server.MapPath("~/logo.jpg"));

// Get logo placeholder position
float[] logoArea = fields.GetFieldPositions("Logo");

// Get logo rectangle
iTextSharp.text.Rectangle logoRect = new Rectangle(logoArea[1], logoArea[2], logoArea[3], logoArea[4]);

// Set logo position in the placeholder (right alignment)
logo.ScaleToFit(logoRect.Width, logoRect.Height);
logo.SetAbsolutePosition(logoArea[3] - logo.ScaledWidth + (logoRect.Width - logo.ScaledWidth) / 2, logoArea[2] + (logoRect.Height - logo.ScaledHeight) / 2);

// Add image
overContent.AddImage(logo);

4) How to add a barcode? Barcode is similar to image:

// Get content to make changes
PdfContentByte overContent = stamp.GetOverContent(1);

// Create a barcode
Barcode39 code39 = new Barcode39();
// Assign barcode value
code39.Code = barcodeValue;
code39.StartStopText = false;

// Create image from the barcode
iTextSharp.text.Image image39 = code39.CreateImageWithBarcode(overContent, null, null);

// Get barcode image placeholder
float[] barcodeArea = fields.GetFieldPositions("AppIDBarCode");
iTextSharp.text.Rectangle rect = new Rectangle(barcodeArea[1], barcodeArea[2], barcodeArea[3], barcodeArea[4]);
image39.ScaleToFit(rect.Width, rect.Height);
image39.SetAbsolutePosition(barcodeArea[1] + (rect.Width - image39.ScaledWidth) / 2, barcodeArea[2] + (rect.Height - image39.ScaledHeight) / 2);

// Add barcode image
overContent.AddImage(image39);

I still have several issues to solve, such as how to put rich text into Text Field. But for now, I have a good start for PDF generation. :)

Relax with Taiji -- for programmers like me

I went to doctor's office this morning and saw a poster to prevent heart disease. It says right things like low-fat diet, no smoking, no alcohol, and regular exercise (with a gym picture). But one important point is missed -- Relax. :)

Everybody has experience of blood pressure difference when he/she is in anger and in sleep. Emotion is a big factor for health, so calm your body and mind down, and relax...

Who does not know how to relax?! "I can lie down on sofa and watch TV" ... Is that a good way to relax after sitting in front of computer for a whole day?

One old phrase says "People's leg gets old first". I am not so old, but my sensitive body is verifying it: After a whole day tiring computer work and when I use eyes to look at computer or TV at night, sometimes I even feel my eyes poll energy from feet to upper body! Our brain and eyes are consuming most of our energy everyday.

So to get heath back, we should:

1) Calm down eyes (no computer or TV) and mind (no work at home)
2) Exercise in slow motion to build energy
3) Let energy go down to get legs stronger

Those are some important values of Taiji to programmers.

If you have no chance to learn Taiji with a good teacher, you can relax by yourself: Stand still and calm your body down (for half an hour). You should feel your feet are heavy and your upper body is light -- that is also Taiji. :) Your feet are heavy because your body is relaxed and energy goes down. Gradually, your feet and legs will become stronger and healthier.

To Generate PDF on the fly

PDF is a standard format for B2B or B2C process these days. I think you may have tried to download bank statement in PDF format. On web sites, the PDF documents are generated on demand. They are not generated and saved somewhere ahead.

I am doing research on how to generate PDF on the fly. I tried some products on market, but I have not found any software that can do the job for me.

This is my requirement: I have some Word template documents with merge fields that can be used to bind database data to generate real documents.
  • I already have a back-end system to generate those Word documents and convert to PDF format to print
  • The front-end system also wants to generate PDF documents on the fly when user clicks a "Get PDF" button on web page
It is too slow to convert Word document to PDF format. For some relatively large document, it may take minutes to finish. This process can be used in back-end system asynchronously, but it is not for web site.

For web site, I have to use PDF forms with form fields to bind database data. I can use PDF editor software to add form fields. But if I create both Word templates and PDF forms, it is hard to maintain both versions.

Is there a software to convert Word templates to PDF forms and to convert Word merge fields to PDF form fields? I tried Adlib, Adobe, ActivePDf, even OpenOffice. Unfortunately, none of them can do merge field conversion.

Why did we use Word template in the beginning? Because Word is better than PDF editor in formatting, and also people are familiar with Word than PDf editor too.

So, it seems I have to create and maintain both Word template (for back-end system) and PDF form (for front-end system) versions. :(

Biztalk Server 2006 and Windows Workflow Foundation: Better Together

This is a short note from last night's MICUG meeting about Biztalk and WF.

A typical system is like below:

Customers -- Processes -- Services -- Data Library -- Database

It is relatively easy to create Database, Data Library, Service programs, and Business Process Programs.

Developer can use
  • Biztalk to model business process across heterogeneous applications
  • Windows Workflow Foundation (WF) to model business processes involving human intervention

But:
  • How should I manage or control the state of a business process involving systems and people?
  • How do I allow business users to modify the business process on demand?

Biztalk 2006 does not allow to modify process on demand. Business users have to ask developer to change, compile and deploy new orchestration assembly. But WF allows to change or create workflow on demand.

Biztalk is not designed for user intervention either, while WF has sequential and stateful workflow to support user intervention.

So we can combine Biztalk and WF together to make everybody happy in two ways:
1. WF accesses Biztalk

This is a common design. WF deals with user interaction to fill in forms, then WF sends request to Biztalk to complete process across other systems.

2. Biztalk accesses WF services

In this design, Biztalk sends Web Service request to WF service for business users intervention. Also business users can access WF program to get Biztalk statistics through Biztalk BAM.

Although Biztalk R2 will build WF engine inside, the above two kinds of integration will still be reasonable.

A difference between OO method call and web service method call

Nowadays, .NET framework is making web service call similar to local method call to developers. There is no much learning curve to write integrated/distributed software using web service.

Traditionally when we develop OO software, we can define base class in API and call that API by passing subclass object. The benefit of this design is that we can use a single API to process multiple subclasses, which is easy and reasonable.

How about the API for Web Service method then? Can we follow the same style (ie. passing subclass object to the API) to define base class in API?

Unfortunately, no!

For example: There is a web method HelloWorld() like below

public class BaseClass
{
public string Name = "Base Class";
}

public class SubClass : BaseClass
{
public string Type = "Sub class";
}

[WebMethod]
public string HelloWorld(BaseClass oClass)
{
return "Hello " + oClass.Name;
}

1. If client gets web service definition through WSDL, it cannot get definition of SubClass. So it has no way to create a SubClass object to call HelloWorld()

2. If client shares BaseClass and SubClass library with web service, then the client can create a SubClass object (oSubClass). But the client can not call HelloWorld(oSubClass) that generated by WSDL either, because WSDL does not have any information for SubClass, not to mention the relationship between SubClass and BaseClass

3. If client shares BaseClass and SubClass library with web service, and call System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke() directly:

Invoke("HelloClass", new object[] {oSubClass});

That still can not work: XmlSerializer will throw exception because we are passing an incompatible object.

So when we design the interface of web service, we have to define APIs separately for each kind of subclass!

ASP.NET TreeView shows old datasource

I do not know what's wrong in my code: The TreeView always shows the old data source, although I assign a new data source to it.

My purpose: To show XML in a tree view. Each leaf element is in "element_name = element_value" format:

name
|
+-- first = F1
|
+-- last = L1

Here is a sample code:

Add a TreeView control to Default.aspx:

<asp:TreeView ID="xmlTreeView" runat="server"
OnTreeNodeDataBound="XmlTreeView_OnTreeNodeDataBound"
ShowLines="True" EnableViewState="False">

In code behind:

public partial class _Default : System.Web.UI.Page
{
List<string> xmlStrList = new List<string>();

protected void Page_Load(object sender, EventArgs e)
{
// Initiate XML data
xmlStrList.Add("<name><first>F1</first><last>L1</last></name>");
xmlStrList.Add("<name><first>F2</first><last>L2</last></name>");
xmlStrList.Add("<name><first>F3</first><last>L3</last></name>");
xmlStrList.Add("<name><first>F4</first><last>L4</last></name>");

// Get random XML string
int i = DateTime.Now.Second % xmlStrList.Count;

// Create XmlDataSource
XmlDataSource dataSource = new XmlDataSource();
dataSource.Data = xmlStrList[i];

// Bind XMLDataSource to tree view
xmlTreeView.DataSource = dataSource;
xmlTreeView.DataBind();
}

// To show leaf element in "element_name = element_value" format
protected virtual void XmlTreeView_OnTreeNodeDataBound(Object sender, TreeNodeEventArgs e)
{
if (e.Node.DataItem is XmlElement)
{
XmlElement element = (XmlElement)e.Node.DataItem;
if (!element.FirstChild.HasChildNodes)
{
e.Node.Text = e.Node.Text + " = " + element.InnerText;
}
}
}
}

Everytime when I refresh the page, I get different XML string data source, but I see the SAME tree view data in browser! --- That is weird ...

I don't know why?

[Update] Copied from asp.net forum:
------------------------------------
This looks like a bug in the XmlDataSource control where modifying the Data property does not invalidate the cache. Setting EnableCaching=false is the workaround to use here.

Thanks,

Eilon
------------------------------------

XmlSerializers.dll?

After I generated web service proxy using WSDL file and ran my web service client, I noticed this log:"Resolver failed to load assembly 'myAssembly.XmlSerializers.dll'". What is that?

I did not use any XmlSerializers DLL reference, or even did not know its existence until I saw XML Serializer Generator Tool:

The XML Serializer Generator creates an XML serialization assembly for types in a specified assembly in order to improve the startup performance of a XmlSerializer when it serializes or deserializes objects of the specified types.

Anyway, I added this line to "Post-build event" in Visual Studio project to generate myAssembly.XmlSerializers.dll dynamically:

"C:\Program Files\Microsoft Visual Studio 8\SDK\v2.0\Bin\sgen" "$(TargetPath)"

propertySpecified in XML Serialization

What will be saved when an object of SomeDate class is serialized?

public class SomeDate
{
private short yearField;
private bool yearFieldSpecified;

public short Year
{
get
{
return this.yearField;
}
set
{
this.yearField = value;
}
}

[System.Xml.Serialization.XmlIgnoreAttribute()]
public bool YearSpecified
{
get
{
return this.yearFieldSpecified;
}
set
{
this.yearFieldSpecified = value;
}
}
}

The serialization code is:

private static void InitializeDateFile()
{
SomeDate oDate = new SomeDate();
oDate.Year = 1910;
oDate.YearSpecified = true;

XmlSerializer xmlSer =
new XmlSerializer(typeof(SomeDate));
XmlTextWriter writer =
new XmlTextWriter(@"SomeDate.xml",
Encoding.Default);
writer.Formatting = Formatting.Indented;
xmlSer.Serialize(writer, oDate);
writer.Close();
}

With oDate.YearSpecified = true;, the output is:

<SomeDate xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<Year>1910</Year>
</SomeDate>

But without oDate.YearSpecified = true;, the output becomes:

<SomeDate xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema" />

Where is Year element?

Here I found the reason:

The XML Schema Definition Tool (Xsd.exe) occasionally generates the XmlIgnoreAttribute when creating classes from a schema file (.xsd). This behavior occurs because value types cannot be set to a null reference (Nothing in Visual Basic), but all XML data types can be. Therefore, the tool creates two fields when it encounters an XML type that maps to a value type: one to hold the value, and another, special field that takes the form of fieldnameSpecified, where the fieldname is replaced by the name of the field or property. Notice, however, that this special field is generated only when the schema specifies that the element need not occur (minOccurs = "0") and that the element has no default value. The XmlSerializer sets and checks this special field to determine whether a value has been set for the field or property. Because the special field must not be serialized, the tool applies the XmlIgnoreAttribute to it.

Good Videos (Taiji)

Although I have practiced Chen-style Taiji for several years, I did not learn Taiji Push-Hand.

From last month, I began to learn Push-Hand with Yang-style Taiji teachers. Their teaching also changed my feeling that Yang-style Taiji is eaiser than Chen-style. With good teachers around, I am confident that I can improve my health and Taiji skills from now on. :)

There are some good videos about Chen-style and Yang-style Taiji here. I save the link in this post for my later reference.

DateTime serialization in .NET 1.1 and 2.0

When a system is running in different time zones, we need pay attention to DateTime field, because .NET 2.0 web service is not compatible with .NET 1.1 web service in design for DateTime field.

.NET 2.0 DateTime uses two-bit flag to indicate time zone: Local, UTC, or unspecified (default)..NET 1.1 does not use that flag.

.NET 2.0 serializes DateTime with time zone information for “Local” and “UTC” DateTime object. There is no time zone for “Unspecified” DateTime object. By default, .NET program uses “Unspecified” DateTime value as local time. In distributed system across time zones, this may bring troubles.

.NET 1.1 always serializes DateTime with time zone.

Below is an example tested in our project:
--------------------------------------------------

public class DateTimeSample
{
private System.DateTime m_dtDateTime =
DateTime.Parse("08/10/1971");

public System.DateTime LocalTime
{
get
{
return m_dtDateTime.ToLocalTime();
}
}

public System.DateTime Default
{
get
{
return m_dtDateTime;
}
}

public System.DateTime UniversalTime
{
get
{
return m_dtDateTime.ToUniversalTime();
}
}
}

.Net 1.1 serialization:

<?xml version="1.0" encoding="utf-16"?>
<DateTimeSample xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<LocalTime>1971-08-10T20:00:00.0000000-04:00</LocalTime>
<Default>1971-08-11T00:00:00.0000000-04:00</Default>
<UniversalTime>1971-08-11T04:00:00.0000000-04:00</UniversalTime>
</DateTimeSample>

.Net 2.0 serialization:

<?xml version="1.0" encoding="utf-16"?>
<DateTimeSample xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<LocalTime>1971-08-10T20:00:00-04:00</LocalTime>
<Default>1971-08-11T00:00:00</Default>
<UniversalTime>1971-08-11T04:00:00Z</UniversalTime>
</DateTimeSample>

When a system becomes distributed …

When we build a system, normally we design a simple framework at first. Then we will add more features to it. When the system is growing bigger and distributed, you will find some weird and interesting bugs.

One example is about date time. When a system was designed at the beginning, the developers decided to use local time to save all the time fields, including “date of birth”. When the program was running locally, there was no problem. But when a call center using Windows Forms application became part of the system, a weird bug arose: some clients’ date of birth had one day difference between sub systems (e.g. 08/18/1966 in one database, but 08/19/1966 in another database). Because of wrong DOB, the clients could not pass credit check.

I spent the whole afternoon and evening yesterday to check code and logs, but I didn’t find any reason. Then before I went bed last night, suddenly I realized the bug was caused by time zone difference in computers.

For example, the business layer server is in “-5” time zone, but a Call Center agent’s computer is set to time zone “-4”. When the agent inputs DOB “08/18/1966”, the Call Center Windows Forms program will pass that date (“08/18/1966 00:00:00”) in local time format to business layer server. Because of time zone difference, the server will receive “08/17/1966 23:00:00” instead – one day difference occurs.

Some computers may have random time zone settings. The DOB field becomes messy.

A quick, temporary, logically (not physically) simple solution: to make all machines in same time zone. For the long term solution, the system should change to use UTC, not local time. That will require longer time to change database and code.

State Machine Workflow: Screencast

Although there are many webcasts or articles about MS Workflow Foundation, it is hard to find good stuff about state machine workflow. So when I found this good introduction of state machine workflow, I was quite happy to watch it several times. :)

Load Balancer and System.Net Connection Management

When you make first HttpWebRequest from you application to access a web site or web service, you will use this code:

WebRequest request = WebRequest.Create(url);
WebResponse response = request.GetResponse();


The process of creating a connection with remote application is complex. In order to improve connection speed for later web requests, .NET framework creates a connection pool for each URL. When a web response comes back, the connection is still available for other web requests with the same URL.

By default, the connection’s KeepAlive attribute is true. But the connection can not stay forever, of course. In .NET 2.0, you can define MaxIdleTime -- when the connection has been idle for more than MaxIdleTime, the connection will be closed by framework.

The logic looks good so far. But when a load balancer is involved, a problem may arise. Load balancer directs a request to a Web service and creates connection for the request. If the connection is idle for a period, the load balancer will break the connection without sending notification out.

Let’s suppose the load balancer breaks idle connection after 90 seconds. But on application server side, .NET connection pool keeps idle connection for 100 seconds (by default). When a new web request is sent at this point, an HTTP transport exception will be thrown.

So it is important to make sure the connection timeout settings. After contacting system administrator for load balancer setting, developer can set MaxIdleTime for the connection:

request.ServicePoint.MaxIdleTime = maxIdleTimeValue;

Please make sure the .NET connection MaxIdleTime must be smaller than the load balancer's max connection idle setting. In this way, .NET framework will create a new connection when the connection has been broken by load balancer.

Service Factory


Patterns & Practices group is making Service Factory to ease Service-Oriented programming in Visual Studio. Don Smith had an excellent webcast to demonstrate how to use Service Factory Visual Studio extension to develop ASMX and WCF solutions.

The Service Factory allows developer to write contract-first services easier (instead of current implementation-first development). Don Smith showed how to use XSD file to generate data contract classes and how to define service contract to generate web service classes.

.NET Guidance Explorer

Although I have disks about .NET patterns & practices, a desktop Guidance Explorer is quite handy:


The only problem is that the explorer does not include enough topics at this point. The developing group is adding and releasing more content nearly weekly.

Is there an auto update feature in the explorer? Unfortunately, no.

Using I/O Completion Port (IOCP) to implement our own thread pool

Here I talk about another old but important technology for .NET server application.

Before we discuss thread pool, a concept should be clear: You can generate as many threads as you want in .NET code, but the default thread pool has at most 25 threads.

For a server application, it is normally necessary to use thread pool to leverage client requests, even on a single CPU computer.

1. Thread pool background knowledge

Creating thread is expensive: To create a thread, a kernel object is allocated and initialized, the thread's stack memory is allocated and initialized, and Windows sends every DLL in the process a DLL_THREAD_ATTACH notification, causing pages from disk to be faulted into memory so that code can execute. When a thread dies, every DLL is sent a DLL_THREAD_DETACH notification, the thread's stack memory is freed, and the kernel object is freed (if its usage count goes to 0).

Thread pool keeps some idle threads in suspended state to save time. At first, there is no thread in the thread pool. When the first request comes, a thread is created to process the request. If the request process takes long time (such as accessing web service), the thread enters sleeping mode at a certain time, the thread pool can create another thread to serve queued requests. When web service response comes back, the sleeping thread is woken up to continue the process. After the request is processed, the thread does not simply die, but serves other requests.

If we do not use thread pool, we will end up using one single thread to serve all client requests sequentially (for long-running request, the performance is terrible); or we create/destroy a thread for each client request, which is time consuming for large amount of requests.

2. Why do we need our own thread pool

By default, there are 25 threads in the thread pool for a .NET application. Normally, the .NET thread pool is OK if there are not many services running in the application and each request process is short. But if the application have many tasks to do (with several back ground threads, scheduler, many long-running requests), it is necessary to build our own thread pool.

3. How to build thread pool

Windows Server I/O Completion Port (IOCP) is a kernel queue that can be used for thread pool. The basic idea is to allocate a number of threads to wait on the queue. A request on the queue includes request data address that is supposed to indicate delegate value, so that the thread can run the delegated method.

You can read more about IOCP thread pool implementation in C# from Part I and Part II

The problem of the article is that the author did not show how to get the request data. You should use Target property of GCHandle to get the real .NET data back.

Microsoft Identity Integration Server

Last night, I attended MICSUG user group meeting. The topic of the meeting was about Microsoft Identity Integration Server (MIIS) to integrate user identities from different data sources. MIIS is a good server for a big company where some legacy systems or departments want to share user identity.

The picture below is an example of MIIS use case. MIIS developer can define attribute mapping for data sources (SQL server, flat file, Active Directory, etc) to import/export identity data.

The current version has limited ability to manage password though. It is understandable that many systems use hash code to store password, so that MIIS can not get original password out. MS is developing password management in next version.

The next version of MIIS can also support Extranet for B2B/B2C identity integration. InfoCard (CardSpaces) will also be supported. It sounds like MIIS will become a good middleware for a big company.

Web Service Host Application Implementation

I have been curious for a long time about how Web Service host application is implemented. After I build .NET web service assembly, how can Web Service host application receive request and call the correct method?

From the research I did on Web Service topics these days, I have a concrete idea about how to implement a Web Service host environment. How does ASP.NET platform process Web Service request is beyond my current ability. Here I only talk about the main steps for a normal Web Service host application to process Web Service request.

Reflection is the key for Web Service host application to work.

1. We define a Web Service class and Web Methods by using according attributes. When we deploy the assembly to the host application, the application uses reflection to load the assembly and initialize properties using those attributes:

Assembly assem = Assembly.LoadFrom(assemblyPath);
foreach (Type t in assem.GetExportedTypes())
{
foreach (MethodInfo info in
t.GetMethods(BindingFlags.Public | BingFlags.Instance)
{
if (info.IsDefined(typeof(WebMethodAttribute), false))
{
// Set according properties
}
}
}

Reflection is also used here to generate WSDL file by going through WebMethodAttribute.
2. When a SOAP request comes, the host application parses the SOAP packet to get class name, method name and parameter list.
3. The host application allocates an instance using the class name, either from instance pool or using a singleton
4. The host application finds the method by using method name and calls the method:

MethodInfo info = type.GetMethod(methodName);
object result = info.Invoke(instance, parameters[]);

5. Return the object in SOAP packet if there is no error; otherwise, return exception

The steps above are my simplest design of a Web Service host. I will consider WSE later.

Dynamic Code Compilation

Maybe you already know this old technology -- dynamic code compilation, but I didn't touch it until today.

Suppose we put code on production box, later we need trace a bug in production environment. Of course there is no Visual Studio installed on production box. How can we do? Sometimes it is too complex to build a testing project and upload to production box. What about building a generic editor with code compilation capability, so that we can put testing code in it and compile the source code dynamically without complex command line typing?

Another example is Application Server. It is a good feature to compile testing code inside the server environment for module's stress test.

Inside the editor, we can define our own format for DLL references. For example:

//@ref "DLL file name"

The editor compiles source code in these steps:
  1. Create an instance of CodeDomProvider CSharpCodeProvider (VBCodeProvider for Visual Basic)
  2. Provide CompilerParameters for compiler options, such as adding DLL references
  3. Compile source code using CompileAssemblyFromSource method of the CodeDomProvider
  4. Check CompilerResults
  5. Execute generated application if there were no errors

SoapHttpClientProtocol and XMLSerializer

I took it for granted that because .NET web service client sends SOAP request to invoke web service method, the client side should use SoapFormatter to serialize SOAP request parameters. Today I know I was wrong.

Actually the client side uses XMLSerializer (instead of SOAPFormatter) to serialize SOAP request content.

Why? Because SOAPFormatter (or BinaryFormatter) serializes all public, private data members and even methods to a stream, while XMLSerializer only serializes public data members. Web service is supposed to integrate separate applications in different platform (.Net, or Java, etc.), so it does not make sense to send private members and methods to the other side.

If you want to send the whole object to the other side, you should use .NET remoting where you can make use of SOAPFormatter or BinaryFormatter.

What I learned in Mid-Atlantic Code Camp in Reston

Yesterday I attended .NET Code Camp in Reston, VA. Before I went there, I had wondered how well it would be, because many good speakers went to TechEd 2006 in Boston. If this code camp had been scheduled days ahead of TechEd, maybe some TechEd speakers would attend the code camp to practice their topics. "Anyway, I will go there to have a look on this beautiful Saturday", so I went there in the morning.

According to the session schedule, some good speakers were not there of course, but I did find several good speakers on the list! :)

The schedule included five tracks: Web track, Data track, Smart Client track, Miscellaneous track, and Security track. There were no much stuff on WinFX (.NET 3.0). Below are the sessions I attended:

1) "Enterprise Library and Data Security": Gary Blatt was still humorous. It's pleasure to listen to his speech. He did not talk about Enterprise Library 2.0 though.

2) "Secure Click Once Smart Client Deployment": MS Regional Director - Brian Noyes' speech was full of exciting technology to me! :) He showed his broad knowledge on .NET platform. Actually, it was the first time for me to see a real "Click Once" (or Click Twice) deployment.

In his speech, Brian showed deployment and application manifest files for an assembly. Whenever an assembly is deployed, new manifest files will include hash code for its XML content, and application manifest file also includes hash code for DLL files. In this way, it is quite difficult for hackers to replace DLL files or to change manifest content.

On user side, Smart Client application runs code according to deployment server URL, so that user can run applications side-by-side deployed from QA server and production server.

It is impossible for me to write down all what I learned from this session. I will wait for his coming MSDN Online article about Secure Click Once Deployment.

3) "Refactoring: Why? When? How?": C# MVP Jonathan Cogley is also one of my favorite speakers (I attended two of his sessions :>). He did not prepare PowerPoint slides. What he did was to show in Visual Studio how to make existing code better for maintenace and performance purposes using refactoring techniques (Rename, Extract Method, Move Method, Introducing Explaining Variable, etc.) and tools (e.g. ReSharper, NUnit). To see a smart guy changing code step by step is really a good learning experience! :)

4) "Web Applications Security: Greatest Hits": Jonathan Cogley demonstrated SQL Injection and Cross-Site Script Attack in ASP.NET application, and how to change code to avoid attacks. Some concepts were not new to me, but I still got some good hints.

For example, we may separate input pages with HTML editor from other input pages. HTML editor accepts Java Script in the textbox, so we should disable Request Validation for that page. For other input pages, we should enable ASP.net Request Validation to avoid script attack.

5) "SQL Server Integration Services with Team Systems": Andy Leonard had planned to show Team system, but unfortunately his VPC died at that time. Instead, he showed us more exciting SSIS feathers. What surprised me in SSIS was its step-by-step debug capability inside Visual Studio.

Andy mentioned that DBA should be involved in Software Development Life Cycle to make system better. I totally agree with him. Nowadays, many systems are designed without DBA, Tester, even Developers being involved --- How can they develop the system without misunderstanding?

6) "Building Ajax Style Applications using ASP.NET 2.0 and Atlas": MS Regional Director Vishwas Lele showed some cool features of Atlas. He made it clear that Atlas does not use ASP.NET 2.0 Callback feature, it uses a special HTTP Handler to process JSON request directly without going through the whole ASP.NET page cycle.

Overall, although I did not see topics about latest .NET 3.0, I still learned a lot from this code camp and went back home happily! :)

.NET Application Server

"Application Server" has been a common buzz word in Java world for many years. J2EE Application Server hosts Java applications and provide environment for deployment, configuration, transaction, logging, session management, instance management, reporting, exception handling, load balancing etc. What application developer cares is mainly the application's business logic.

Application Server in Java world makes perfect sense, because Java runs on different operating systems. It is necessary to build a common environment to hold applications.

But in .NET world, "Application Server" is not well-known. We do not need Application Server in .NET, right? Yes or no.

Yes. The main reason is that .NET is built upon Windows system, so that .NET can use features provided by Windows System (e.g. Transaction, Logging) directly.

No. We still need an integrated environment for Enterprise software. That is the reason why MS Enterprise Library exists. But to set up and use that library is still complex for a normal developer. Compared with Java Application Server environment, .NET framework is not an integrated product for developers and project managers. The only one Application Server I know is Interactive Server. From my research on that product these days, it has some limitations.

So, can we make an application server product to hold various .NET applications? Technically yes, although not easy. Maybe MS will build application server later. I heard a rumor that IIS 7 is a kind of Application Server, but I doubt. :)

Social Engineering, the USB Way

The article "Social Engineering, the USB Way" amused and scared me a lot: When a credit union’s employees happened to find USB keys (with Trojans software), they were so happy to pick up and plug into company computers --- Everybody likes free stuff! :) The Trojans software ran secretly on those computers and sent emails to hackers with users’ important data --- It is so easy to hack a system!

From that story and discussions, I realized two important security problems people ignore:

1) Most people use Administrator account for daily work.

For a financial company with customer SSN, birth date, and address in database, it is quite important to train employees to use non-admin account to avoid virus and Trojans software to some degree. However, in reality, many people (even IT people!) in financial company still do not feel that danger. They love the convenience of Administrator account!

2) Auto-Run feature may run malicious software automatically

Auto-Run feature is useful to play music CDs (not from Sony!), but if hackers use that feature to install virus or Trojans software, it will become a nightmare to users.

Great .NET posters

[Change note: The URL of those .NET posters has been changed]

"Barone, Budge & Dominick" has a great collection of posters relating to.NET technology. You can find the first page Here.

.NET web service tips

Here are list of tips for .NET web service development. I put here so that I need not google to find them later. :)

1) Generate web service interface so that we can separate implementation from WSDL. When WSDL is changed someday, we will not lose our implemenation code:
wsdl.exe /si ServerInterfaceSample.wsdl

2) Web service can use ASP.NET session state. To enable ASP.NET session state, set WebMethod.EnableSession = True. On the client you must also create a new instance of System.Net.CookieContainer. Note that to persist data for the entire application, neither step is required.

Client code:

//On the client you must add the following code (for session state only):
serviceName.CookieContainer = new System.Net.CookieContainer();

Server code:

//Setting EnableSession to true allows state to be persisted per Session
[WebMethod(EnableSession=true)]
public String UpdateSessionHitCounter() {
//update the session "HitCounter" here
return Session["HitCounter"].ToString();
}

3) Asynchronous call to web service:
Use the event-based model to asynchronously call your Web services.

//First implement the HelloWorldCompleted method using the following signature:
//public void HelloWorldCompleted(object sender, HelloWorldCompletedEventArgs args)

//Create the Web service
HelloWorldWaitService service = new HelloWorldWaitService();
//Add our callback function to the event handler
service.HelloWorldCompleted += this.HelloWorldCompleted;
//Call the Web service asynchronously
service.HelloWorldAsync("first call");
//when the Web service call returns the HelloWorldCompleted method will be called

Optimistic Concurrency Control using RowVersion in Sql 2005 and C#

To use optimistic concurrency control, one option is to add a RowVersion (kind of timeStamp) column in Sql 2005 table. Its value will be modified automatically by Sql server whenever the row is updated.
alter table dbo.authors add LastRowVersion RowVersion

The type of the newly added column is "TimeStamp" in Sql 2005, but its value has no relationship with time at all. The RowVersion column only uses the same structure of TimeStamp to save version number (8 bytes).

The process of optimistic concurrency control using RowVersion column is like below: When user A gets a record and tried to edit, the RowVersion value is also got to A's program. Another user B also gets the same record to edit, but B saves his/her content before A saves. Now the record's LastRowVersion is changed. When A tries to save content into the record, the save process should compare the previously fetched LastRowVersion with the latest value in the database. If the two values do not match, user A should get error message like "Another user has saved the content in front of you. Your content cannot be saved", blah blah.

Now, let's see how to deal with the RowVersion column in ASP.NET/C# code. In data access layer, we still use SqlDbType.Timestamp for RowVersion to transfer value between our code and Sql Server. But in other layers, we should use byte[] array.

When the ASP.NET page gets the data from business layer, the RowVersion value can be saved in ViewState:
ViewState["RowVersion"] =
Convert.ToBase64String(bo.LastRowVersion);

When user clicks "Save" button in edit page to save changes, we can get the old RowVersion from ViewState in postback:
byte[] rowVersion =
Convert.FromBase64String(ViewState["RowVersion"] as string);

Then we can pass the rowVersion variable with the new content to data access layer for concurrency check as mentioned above.

In some applications, ViewState may be disabled. In this case, we can save the value in ControlState, which needs several more lines of code than using ViewState.

Mix'06 sessions are online now :)

Luckily I attended Microsoft Mix'06 conference in Las Vegas in March, 2006. As the conference included various topics covering web design, web development, smart client application, next generation of OS and browser, so the conference was named Mix (mixture).

I was quite impressed by the demos of Windows Presentation Foundation, Windows Presentation Foundation Everywhere, and Atlas. I think other sessions must be also interesting, such as "building your own search engine", but I couldn't attend those meetings at that time.

But now, I can watch all those sessions online :)

Management and Engineers

I saw this article from Sahil Malik's blog. It is quite what I am feeling for current IT industry.


A man in a hot air balloon realized he was lost. He reduced altitude and spotted a woman below. He descended a bit more and shouted, "Excuse me, can you help me? I promised a friend I would meet him an hour ago, but I don't know where I am."

The woman below replied, "You are in a hot air balloon hovering approximately 30 feet above the ground. You are between 40 and 41 degrees north latitude and between 59 and 60 degrees west longitude."

"You must be an engineer," said the balloonist.

"I am," replied the woman, "How did you know?"

"Well," answered the balloonist, "everything you told me is, technically correct, but I have no idea what to make of your information, and the fact is I am still lost. Frankly, you've not been much help so far."

The woman below responded, "You must be in Management."

"I am," replied the balloonist, "but how did you know?"

"Well," said the woman, "you don't know where you are or where you are going. You have risen to where you are due to a large quantity of hot air. You made a promise, which you have no idea how to keep, and you expect people beneath you to solve your problems. The fact is you are in exactly the same position you were in before we met, but now, somehow, it's my fault."

Middle-Ware is in the Middle

I attended the "Capital Area Microsoft Integration and Connected Systems User Group" last night. It had good topics about lessons learned from Biztalk integration projects.

One of the lessons I feel important is to really know "Middle-Ware is in the Middle", which means whenever there is something wrong with the operation of an integrated system, normally either developers or clients will complain:"Middle-Ware is not working properly". But in fact, most of the cases are not because of the Middle-Ware, but the end-point applications.

To protect from the easy blame from other people, tracking and reporting are the life-savers: Keep all detailed tracking and reporting information to indicate where the real problem is, e.g. the schema of the source data has been arbitrarily changed.

We needn't work on really big project to create/use Middle-Ware. Even for small project, we will write middle-ware program occasionally. For example, we may write either Windows Service to collect data or Web Service to process data. For end user, they are back end Middle-Ware. It is important to keep event log for each process.

Service-Oriented Architecture and Biztalk Server

When people talk about Service-Oriented Architecture (SOA), they talk more details about how to write Web Services. Actually, SOA is more than web services. For developer, SOA means contract-first development for a distributed system to support interoperability between applications.

As Biztalk Server 2006 is released, let's look at its architecture for an example of SOA.
Biztalk messaging architecture
Biztalk server includes four parts: Receive Port, Message Box, Orchestration, and Send Port. Receive Port uses protocol (FTP, SOAP, etc.) specific Adpater to receive file or message, then persists the (transformed) message into Message Box. Message Box activates according Orchestration or Send Port based on subscription of the message type. Orchestration is a process workflow to do some work based on incoming message, then Orchestration may persist a new message in Message Box. Message Box activates according Send Port to send the new message out.

From the architecture, we can know that Biztalk server is in message-based. Its main function is to receive a message, and to send message after doing some work. You can also take Biztalk as a message processing service.

For development, Biztalk developers typically start by modeling the messages using XML schema. Developers then promote several message properties for routing purposes. Then Developers configure Orchtestration or Send Ports to subscribe the messages matching those promoted properties. The orchestration developed also deals with XML schema to process message. From these steps, you can see it is a contract-first development.

Biztalk application is also autonomous. Biztalk does not care about the implementation details of other connected applications. It only cares the messages received/sent using the predefined XML schema.

Until now, you can say Biztalk is a good example of Service-Oriented application. Is it a web service? No, although you can optionally publish an orchestration as a web service.

Biztalk architecture allows flexibility to connect to nearly any kind of legacy applications by using different Adapter in receive/send port. Let's take receive port as an example:
Biztalk receive port
The receive port uses protocol-specific Adpater to receive original message and add other context information to build an internal message, then the pipeline can decode the message, the last step in receive port is XML schema mapping to transform the original message into another format that Orchestration knows.

Developers can use many kinds of Adapter to receive/send message, e.g. using MSMQ, SOAP, even Windows Communication Foundation Adapter. This architecture allows to integrate various applications without modifying those applications' code.

Wrap-Up
Biztalk is a good example of Service-Oriented Architecture application. The architecture uses contract-first development and allows potential extension.

Contract-First Web Service development

Although we normally define interface (contract) between client and server in distributed environment before we really write logic code, Visual Studio does not provide this Contract-First mechanism for Web Service application development.

What VS provides is implementation-first: Developer writes web methods first, then VS generates WSDL (contract) to allow client developer to implement web service client.

The advantage of implementation-first approach is that it is quite easy to develop a web service. Developer need not write complicated WSDL file manually. When web service client accesses the web service, .NET framework will reflect web methods in web service code and generate WSDL dynamically.

The disadvantages of implementation-first approach are:
  • Web service developer can change web method easily, which may change the generated WSDL accidentally or without enough warning. This will break the contract between web service and client.

  • Web service developer may ignore the compatibility of generated WSDL to include complex object type, like DataSet. This will make Java client difficult to get result because DataSet is only used in .NET framework, not Java framework.

So, how can we use current tools to develop web service? Actually, we can use implementation-first approach in the beginning:
  1. We can write web methods in Visual Studio without put logic code inside. Then we can let VS generate the complex WSDL for us.

  2. We examine the WSDL to make sure it does not include .NET specific data types. We may add more parameters to it.

  3. If we changed the WSDL, we can re-generate web service code using "wsdl.exe /server <WebService WSDL file>"

  4. When the contract need modification, we should explicitly update the WSDL file and re-generate web service code. We cannot modify web method code directly. This step may overwrite original code, so it is a good idea to separate implementation code into another file



If we follows the steps above, we should be able to keep interoperability between web service and client.

Windows Workflow Foundation (WF) and ASP.NET

Windows Workflow Foundation (WF) is an exciting addition to .NET development framework, which makes developing enterprise Workflow application easier -- You need not worry about maintaining states for each activity inside WF. As most of the enterprise applications are web-based, I am more curious how WF can integrate with ASP.NET framework.

Let's take a brief look at how WF and ASP.NET work separately.

WF: WF runtime creates a WF instance that is defined by developer using workflow designer or XOML (a XAML configuration file). Then the instance is started to process passed-in parameters (for sequential workflow) or wait for an event to trigger a series of activities (for state machine workflow). To avoid blocking front-end application, by default the WF instance runs asynchronously using another thread. When WF instance is idle (e.g. while waiting for human intervention), the instance is serialized and saved in database. Later, when certain event happens, the instance will be deserialized and continue to process from former state. A sample of WF code is like below:

// Get a reference to the Workflow runtime
WorkflowRuntime wr = WorkflowWebRequestContext.Current.WorkflowRuntime;

// Attach to the WorkflowCompleted event
wr.WorkflowCompleted += new EventHandler<workflowcompletedeventargs>(CallbackMethodWhenCompleted);

// Create workflow instance
WorkflowInstance workflowInstance = wr.CreateWorkflow(typeof(Samples.HelloWorkflow), parameters);

// Start the workflow instance
workflowInstance.Start();

ASP.NET: ASP.NET runs synchronously for each request. There is no relationship between each request. After the request is processed, a response will be returned.

1) The synchronous ASP.NET and asynchronous WF makes integration a little bit complex. If ASP.NET code call WF runtime to create an instance to run without special configuration, ASP.NET code will return directly without waiting for the WF instance to finish -- This is not what we want.

2) When a new request comes, ASP.NET cannot create a new WF runtime. Because WF runtime cannot be loaded more than once in the same AppDomain.

To solve the first problem, uses the <workflowruntime> configuration section in the configuration file to store information to include WorkFlowPersistenceService and WorkFlowSchedulerService. The former service automatically persists the workflow instance to the specified database as soon as the workflow becomes idle. The second service ensure that the ASP.NET thread in charge of executing the current request waits until the workflow is completed, in another word, the service guarantees that the execution of the workflow is synchronous and that the Start() method returns only when the workflow has ended or is idle.

For example, the configuration part can be like below:

<WorkflowRuntime Name="WorkflowServiceContainer">
<Services>
<add type=
"System.Workflow.Runtime.Hosting.ManualWorkflowSchedulerService,
System.Workflow.Runtime, ..." />
<add type=
"System.Workflow.Runtime.Hosting.SqlWorkflowPersistenceService,
System.Workflow.Runtime, ..." />
</Services>
</WorkflowRuntime>

To solve the second problem, WF provides WorkflowWebRequestContext to provide unique runtime instance in an AppDomain (see above code example).

CTE in SQL Server 2005

CTE (common table expression) is a good feature in SQL Server 2005 to make recursive logic easier. But the implementation of SQL Server 2005 has limitation. Let's look at an example as below to get directory tree:

DECLARE @DirID int
SET @DirID = 11;

WITH cte_subDir (dirID, dirName, parentDirID)
AS
(
SELECT dirID, dirName, parentDirID
FROM DirTable
WHERE dirID = @DirID

UNION ALL

SELECT dirID, dirName, parentDirID
FROM DirTable
INNER JOIN cte_subDir
ON DirTable.parentDirID = cte_subDir.DirID
)

IF @DirID > 10
DELETE FROM DirTable
WHERE dirID IN (SELECT dirID FROM cte_subDir)

When I ran the script, I got this error:

Msg 156, Level 15, State 1, Line 18
Incorrect syntax near the keyword 'IF'.

It took me about one hour that I realized CTE With statement cannot be followed by IF statement. It can be followed by SELECT statement though.

To make the above logic, I have to create a temporary table using

SELECT * INTO #tempTable FROM cte_subDir

Then, I can use #tempTable for the remaining logic.

SelectedDate of ASP.NET Calendar control

During years of ASP.NET development, sometimes I find I ignored important page lifecycle even I used some features frequently.

Here is an example of Calendar control. I put the Calendar control dynamically into one page, and put data binding logic for the selected date in Page_Load() like below:

protected new void Page_Load(object sender, EventArgs e)
{
// Call base method
base.Page_Load(sender, e);

// Get selected date
DateTime selectedDate = wsCalendar.SelectedDate;

// Bind data
BindEventList(selectedDate);
}

When I ran the program and check the selectedDate variable, you know what? On the first time postback, the value of selectedDate was 1/1/0001 while I had selected 4/17. On the second time of postback, the value of selectedDate was 4/17, instead of the latest selected date 4/20: I got the selected date of the last time, not the current one!

That result amused me for a while until I realized I made this mistake: I should get the selectedDate in Selectionchanged event. During Page_Load(), the real selectedDate has not been updated yet although part of postBack logic was already executed before load event.

protected void DateChanged(object s, EventArgs e)
{
// I can get the real selected date now
DateTime selectedDate = wsCalendar.SelectedDate;

// Bind data
BindEventList(selectedDate);
}

Try/Catch internal

I was curious how to implement Try/Catch statement in compiler. When an exception occurs in methodChild(), how can OS know which part of code to catch that exception? If methodChild() does not catch the exception, the methodParent() (calling methodChild()) should catch the exception. If methodParent() does not catch, the exception should be handled by parent method of methodParent(), and so on ... But how can this catching process be implemented?

Today, several articles (e.g. Calling Conventions and X64 Primer) let me understand what is going on under the hood.

In Win32, compiler generates special instructions for Try/Catch statement. Every function that needs attention due to an exception must add an element to a thread-global linked list upon entry, and remove it upon exit. Each element in the linked list contains a function pointer to call in the event of an exception, and then some data that said function will consume. When an exception is thrown, OS will walk through the linked list to find a function to process the exception properly.

The linked list structure of Win32 is not efficient. In addition, the linked list actually resides on the stack, thus there is a function pointer (to remove element upon exit?) sitting right below the return address on your stack -- Buffer overruns.

In contrast to the Win32 exception handling, Win64 executable contains a runtime function table. Each function table entry contains both the starting and ending address for the function, as well as the location of a rich set of data about exception-handling code in the function and the function's stack frame layout.

When an exception occurs, the OS walks the regular thread stack to search the runtime function table in that module, locates the appropriate runtime function entry, and makes the appropriate exception-processing decisions from that data.

ASP.NET 2.0 Master Page article

Scott's article style is informative and inspiring. Here is an article about ASP.NET 2.0 Master page tricks and tips. Enjoy :)

In this article, Scott makes the masterpage process clear: Master page replaces the content page's children to itself, then the master page looks for Content control in the controls formerly associated with the content page. When the master page finds a Content control that matches its ContentPlaceHolder, it moves the controls inside the Content control into the matching ContentPlaceHolder. This process happens after the content page'’s PreInit event, but before the content page'’s Init event.

Note: At this time, the Content control itself does not exist in content page's DOM tree any more, which means you cannot use ContentControl.FindControl("ctl") .

In the article, Scott gives good suggestion for interaction between mater page and content page. Basic idea is not to bundle mater page and content page too tightly. It is better to use a separate event, and let the two pages subscribe to the event.

There are also several other useful tips. I believe I will refer to that article later.

WPF/E architecture

WPF/E (Windows Presentation Foundation Everywhere) has subset feature of WPF. It can dynamically arrange document layout, show 2D, subpixel text, etc.

The good thing about WPF/E is it is cross browser and cross OS framework.

The not so good things are:
  • Cannot support 3D
  • Need plugin to browser to show content
Below shows the architecture of WPF/E:


508 Compliant issue in web development

As I am building websites in a government department, it is quite common to hear "Is this site 508 compliant"? Most people in corporate do not know what 508 is about, but in government, 508 makes developers pulling their hair every day.

508 compliance is to let disabled people view websites and low end browser (e.g. text-only browser) display content correctly. If a page displays image, that image must have alternative text to show text information about that image. So disabled user can use screen reader to "view" the page.

But 508 does not mention JavaScript. So can we use JavaScript in web pages? Some people would say No! Why? Because if user disables JavaScript in browser, the web site is not workable for him/her anymore, which means it is not a 508 compliant site. So to make a workable 508-compliant, we have to develop web pages without any JavaScript involved. Of course, we can add JavaScript to make pages look nicer, but without JavaScript, the web pages must also work.

For a web site only displaying content most of the time, it is ok without JavaScript. But for a web application (like Intranet), without JavaScript means driving a car that has no engine. How can you make an Intranet interactive only by using URL parameters and hidden fields? Not to mention good user experience of AJAX that definitely needs JavaScript.

508 was made a decade ago when Internet was supposed to display content, not for interaction. Nowadays, 99% of people will not use text-base browser. 508 seems out-of-date now.

WPF and WPF/E

WPF (Windows Presentation Foundation) is a MS .NET framework to build desktop/browser application. As the name indicates, WPF is to display content in interactive manner, like 2D, 3D, audio, video, etc. Normally, WPF application can be compiled in desktop or browser mode without much code change: because WPF browser application uses XAML (not HTML) to display content.

WPF is based on DirectX. So if you want to write a high-performance application (e.g. 3D game), you had better use DirectX, not WPF. For other cases, WPF will be your choice for next generation Windows development.

The advantage of WPF is that application can take advantage of user's computer resource (memory, CPU, harddisk, etc), and easy to deploy without installation. The latter is the biggest reason why web sites are so popular.

The disadvantage of WPF is it requires .NET 2.0 and WinFX platform on user's computer. So it is limited on Windows machine.

To solve that problem, MS is developing WPF/E (everywhere). It has subset features of WPF (without 3D support), but it can run in IE, Safari, or Firefox. It also uses XAML for content layout.

WPF/E need a small plugin inside browser to run application. WPF/E application includes XAML and .NET (C#, VB.NET) compiled library (IL). For this reason, WPF/E runtime includes a simplified version of .NET framework (~200KB)

WPF/E is much better than Flash in both feature and development.

Hopefully, we can try those technology this year :)

Dragon CPU makes computer industry interesting

These days, Dragon CPU (also named GodSon chip) has been a hot topic in IT websites. The performance of the CPU is similar to lower-level Pentium 4. It is a 64-bit CPU with 95% MIPS compatible instruction set. In 03/2006, a ~$180 Godson II computer running Linux called Longmeng (Dragon Dream) was announced.

With the low price and high performance, that CPU may threaten WinTel: Just like somebody said, M$ does not care Linux, but care Linux + cheap computer!

The IT world will change soon. Let's wait and see ... ...

labeling of insect-derived red food dye

Until today, I didn't know that some "artificial colors" in our food product are from insects (here). I thought I was careful enough to avoid food with animal stuff inside, but now I think I had better be a farmer to get pure vegetarian food.

"Derived from the ground bodies of female cochineal beetles, the colorings are currently used in a variety of ice creams, yogurts, fruit drinks, alcoholic drinks and candy products, to which they bring a characteristic pink, red or purple color."

"
Currently the FDA only requires that the ingredient is labeled as a ‘color added' or ‘artificial color'"

"Why not use a word that people can understand?” said CSPI director Michael Jacobson. ”Sending people scurrying to the dictionary or to Google to figure out what ‘carmine' or ‘cochineal' means is just plain sneaky. Call these coloring what they are – insect-based.”

Alternative Living

It seems like vegetarian is a good choice for us and for animals:

Health Studies in Confusion

According to the so-called 8-year study: low-fat diets do not benefit cancer significantly (here).

When I first saw that result, I doubted my own vegetarian habit: Is it a really healthy to be a vegetarian? I can tell from my years of vegetarian experience that it does not give me much more energy, but compared with when I ate meat and eggs, my health is not getting worse at least. I am getting old as well. It is difficult to tell if vegetarian really improved my health. But from theory, it should be a good eating habit.

Then I read several articles about that study in the coming days. It turns out that study was based on out-of-date hypothesis. It even did not distinguish lard or olive oil!

They spent more than 400 million dollars for that nonsense study.

I will not take those kind of study for granted any more. I will keep my diet because I prove it works for me.

Unhealthy lifestyle

During the past dozen years of work as a software engineer, I met many computer guys who do not look like in their age. They are either too fat, or too weak. The reason is obvious: we sit in front of computer for years without necessary exercise, and even worse, many of us like to eat something during coding.

I am quite sad with this lifestyle although I love computer and already get used to programming life. I wanted to find a way to improve health without interrupting work. Here are what I found useful to me at least:
  1. Stand up and keep knees bent a little bit for several minutes every hour. You can talk with coworkers or design a solution to solve a problem at the same time. Human being gets older from her/his legs, so it is very important to strengthen our legs
  2. Practice TaiJi or QiGong to build energy. Healthy does not necessarily mean big muscle. We needn't go to expensive gym to keep us healthy. Simple QiGong can work perfectly for our body and our mind. For Taiji, I recommend Chen-Style TaiJi that is not so slow.
  3. Eat more vegetable and fruit. Saturated fat is not good for our body.

What else?

The 'using' statement in .Net 2.0

The 'using' statement in .Net 2.0 is to dispose an object implicitly when the code block is finished. The compiler translates 'using' statement to try...finally statement and call the object's dispose() function.

Suppose that we have this using statement:

using (obj)
{
obj.DoSomething();
}
From concept, the compiler translates that block into code like below:

try
{
obj.DoSomething();
}
finally
{
if (obj != null)
{
obj.Dispose();
}
}

But when obj is not an IDisposable object, that using statement can not be compiled. For example, ITestInterface has one method DoSomething(). TestClass is an implementation class for ITestInterface and IDisposable, the code below can not be compiled:

public class Program
{
static void Main(string [] args)
{
ITestInterface obj = new TestClass();
using (obj)
{
obj.DoSomething();
}
}
}

To make it work, we should add "as IDisposable":

using (obj as IDisposable)
{
obj.DoSomething();
}

The logic is same as:

using (IDisposable obj2 = (obj as IDisposable))
{
obj.DoSomething();
}