Strongly Typed Identifier Scheme

Over the years I have tried a number of Identifier schemes:

  • Atomic: String UserId; int UserId; Guid UserId;
    • There is no link to the underlying domain type.
    • There instances are easily confused. I.e. A page taking a Url param of “Id=123” may refer to a User, UserGroup, UserProfile, NTLM, Email, etc. This gets significantly worse in complex enterprise systems, eg. “ServiceLocationIdentifier” may be named as a local variable called “sl” and the developer needs to correctly interpret this.
  • Classic Strongly Typed: class UserIdentifier { string UserID { get; set; } }
    • This is significantly better, but there is the overhead of creating and maintaining all these types
    • There is no scheme in place to unify them. It is possible, ofcourse, to create an class abstract BaseIdenfifier { ... }
    • There is no coupling from the Identifier to the Domain Type, ie User and UserIdentifer as not linked by the scheme. Meaning you would have to build this in with every class

The code of the idea I am playing with is:

public class Id<T>
{
public string Id { get; set;}
public Type DomainType { get; set; }
public override string ToString() { return Id; }
public override int GetHasCode() { return Id.GetHashCode(); }
}

This code as the following properties (none very interesting):

  • The DomainType (T) is strongly maintained.
    • This means that new Id<User>("Bob") == new Id<Group>("Bob") , should throw an TypeMismatch exception
  • You can treat it as a string new Id<User>("Bob") == "Bob") will evaluate as true.
  • There is a string coupling between the Identifier and the Domain Type
    • class User { public Id<User> UserId { get; set; } }

The fun stuff, however, happens when you introduce a ‘Services’ class:

public class static Services
{
public T GetInstance(Id<T> Identifier) { … }
public Uri GetUri(Id<T> Identifier) { … }
public string GetDisplayName(Id<T> Identifier) { … }
}

The result of which are statements like:

public void ExampleCode()
{
Id<User> bob = new Id<User>(“Bob”);
string EmailText = string.Format(“Hello {0}, Please click on the following link to edit your details {1}.”,
Services.GetDisplayName(bob),
Services.GetUri(bob));User bobDetails = Services.GetInstance(bob);
SmtpHelper.SendEmail(bobDetails.Email, EmailText, “Welcome…”);
}

The full version of the class is as follows:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Xml.Serialization;namespace GL.General.Core
{
/// <summary>
/// Encapsulated the relationship between a type and its string identifier.
/// It provides a type-strong relationship, in which for example a UserId and SaleId are NOT comparible
/// </summary>
/// <typeparam name=”T”>The domain type associated with the identifier</typeparam>
public class Id<T>
{
/// <summary>
/// Default Constructor
/// </summary>
public Id()
{
DomainType = typeof(T);
ID = null;
}/// <summary>
/// Strong constructor
/// </summary>
/// <param name=”id”></param>
public Id(string id)
{
DomainType = typeof(T);
ID = id;
}/// <summary>
/// Copy constructor
/// </summary>
/// <param name=”toCopy”></param>
public Id(Id<T> toCopy)
{
DomainType = typeof(T);
ID = toCopy.ID;
}

/// <summary>
/// The string Identifier. The value of the identfier
/// </summary>
[XmlAttribute]
public string ID { get; set; }

/// <summary>
/// The type of the identifier (used for meta-data and validation only)
/// </summary>
[XmlIgnore]
public Type DomainType { get; set; }

/// <summary>
/// This is a helper method, which provides the <see cref=”DomainType”/> which is serialised in XML
/// </summary>
/// <remarks>
/// This attribute, while serialised to XML, is ignored when it is read in
/// </remarks>
[XmlAttribute]
public string Class
{
get
{
if (DomainType == null) return string.Empty;
return DomainType.FullName;
}
set
{
// Ignored in deserialisation
}
}

/// <summary>
/// Provide Equality checking with Id<T> and string
/// </summary>
/// <param name=”obj”></param>
/// <returns></returns>
public override bool Equals(object obj)
{
if (obj == null && ID == null) return true;
if (obj != null && ID != null) return obj.ToString().Equals(ID);
return false;
}

/// <summary>
/// Provide Equality checking with Id<T> and string
/// </summary>
public static bool operator ==(Id<T> lhs, Id<T> rhs)
{
if ((object)lhs == null && (object)rhs == null) return true;
if ((object)lhs != null && (object)rhs != null) return lhs.ID == rhs.ID;
return false;
}

/// <summary>
/// Provide Equality checking with Id<T> and string
/// </summary>
public static bool operator !=(Id<T> lhs, Id<T> rhs)
{
return !(lhs == rhs);
}

/// <summary>
/// Provide Equality checking with Id<T> and string
/// </summary>
public static bool operator ==(Id<T> lhs, string rhs)
{
if ((object)lhs == null && (object)rhs == null) return true;
if ((object)lhs != null && (object)rhs != null) return lhs.ID == rhs;
return false;
}

/// <summary>
/// Provide Equality checking with Id<T> and string
/// </summary>
public static bool operator !=(Id<T> lhs, string rhs)
{
return !(lhs == rhs);
}

/// <summary>
/// Provide Equality checking with Id<T> and string
/// </summary>
public static bool operator ==(string lhs, Id<T> rhs)
{
if ((object)lhs == null && (object)rhs == null) return true;
if ((object)lhs != null && (object)rhs != null) return lhs == rhs.ID;
return false;
}

/// <summary>
/// Provide Equality checking with Id<T> and string
/// </summary>
public static bool operator !=(string lhs, Id<T> rhs)
{
return !(lhs == rhs);
}

/// <summary>
/// ToString will provide the underling string identifier
/// </summary>
public override string ToString()
{
return ID;
}

/// <summary>
/// Use the underlying string as a HashCode
/// </summary>
/// <returns></returns>
public override int GetHashCode()
{
if (ID == null) return 0;
return ID.GetHashCode();
}
}
}

.NET Process Maximum Memory

[MOVED: Updated version at https://home.guylangston.net/blog/article/MaxMemoryDotNetProcess]

 

“This whole article started because of the a memory leak in our Fares Engine…. It has become one of the longest debugs of my career.”

OutOfMemoryException

The maximum allocatable size of a .NET process depends on the following factors:

  • The CPU, which for the .NET universe is
    • x86 which has a 4GB address range, although in practice the actual available memory is around 3.2GB as the BIOS and video share the available address range.
    • x64 which has an address range of 18.45 exabytes which is practically unlimited (although we said that about 386, x86 and y2k)
    • Italium , which is beyond the scope of this article.
  • The Windows operating system
    • Windows 98/ME
    • Windows NT/XP/2000 (x86 and x64)
    • Vista (x86 and x64)
    • Windows Server 2003 (x86 and x64)
  • The headers PE signature of the OS Process /LARGEADDRESSAWARE
  • Available swap space

The x64 platform (x64 CPU, x64 OS) will yield a maximum process size in excess of the available system RAM. For example, my 8GB Windows 7 machine was able to yield a process of 8,656 MB, although at that point the system was swapping so much it was unusable.

The x86 platform (x86/x64 CPU, x86 OS) will yield 1.5GB as the maximum process size be ##DEFAULT##. Most people assume this will be 2GB, however the garbage collector needs its own space and does not allocate the maximum.

It is possible to allocated over 2GB on a .NET x86 process by marking the assembly or EXE with /LARGEADDRESSAWARE. This cannot be done via Visual Studio, although you can create a Post Build Event (see attached solution for an example). Execute the following command-line tool EDITBIN.EXE /LARGEADDRESSAWARE MyApp.exe. This command can normally be found inC:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\bin. In order to confirm that the binary headers have been correctly updated run the following command DUMPBIN.EXE /HEADERS MyApp.exe, which should give:

Microsoft (R) COFF/PE Dumper Version 9.00.30729.01
Copyright (C) Microsoft Corporation. All rights reserved.Dump of file MaxMemory-x86-LargeAddressAware.exe

PE signature found

File Type: EXECUTABLE IMAGE

FILE HEADER VALUES
14C machine (x86)
3 number of sections
49FDF23D time date stamp Sun May 03 20:36:29 2009
0 file pointer to symbol table
0 number of symbols
E0 size of optional header
122 characteristics
Executable
Application can handle large (>2GB) addresses
32 bit word machine

Strangely, it easier to test this on an x64 system which has lots of available memory. In tests (see test utility described later) I have been able to allocate 2’803MB.

It is more complex on a x86 systems as the operating system must have the /3GB switch in the c:\boot.ini file. For example, marking an .NET application with /LARGEADDRESSAWARE, but running it on a Windows XP system will yield 1,500MB max — even if the machine has the maximum physical RAM of 4GB/3.2GB free! You need to have both /LARGEADDRESSAWARE on the specific application, and the operating system configured with /3GB.

.NET Maximum Memory Test Tool for x86, x64

I have found it tricky to isolate the different factors and to marry up the theory with actual hands-on results. To this end I created a utility allocate increasing amounts of memory (in 64KB chunks) to determine the actual maximum allocatable memory for a particular system (CPU, OS, RAM, swap space, and configuration /3GB, /PAE)

Full source (GPL) and compiled .EXE are available for download from the footer of this article. No installation is required.

The utility comes in three flavours:

  • x64 — called "MaxMemory-x64.exe"
  • x86 — called "MaxMemory.exe"
  • x86 + /LARGEADDRESSAWARE — called "MaxMemory-x86-LargeAddressAware.exe"
[MaxMemory.exe screenshot]
Fig. 004 — MaxMemory.exe screenshot

Table of Findings

  • I am able to allocate 8,656MB using "MaxMemory-x64.exe" on x86, Windows 7, 8GM RAM
  • I am able to allocate 2,803MB using "MaxMemory-x86-LargeAddressAware.exe" on x86, Windows 7, 8GM RAM
  • I am able to allocate 1,411MB using "MaxMemory.exe" on x86, Windows 7, 8GM RAM
  • I am able to allocate 2,191MB using "MaxMemory-x86-LargeAddressAware.exe" on x86, Windows XP + /3GB, 2’496MB RAM, VMWare
  • I am able to allocate 2,178MB using "MaxMemory-x86-LargeAddressAware.exe" on Xeon, Windows Server 2003 SE + /3GB, 2GB RAM, VMWare

I am sure the tweaking the allocations and system configuration by yield better results, and that the actual number may also vary from attempt to attempt.

On the x86 platform, allocations above the 1,500MB mark seem to be very slow. I do not know why this is the case.

A very good article, showing various approaches to .NET Reliablity and related defensive techniques.

— Comments and refinements are welcome, Author.

Languages designed to mess with your head

[Moved: https://home.guylangston.net/blog/article/TowerOfBabel]

Anyway, there is a very good picture of the evolution of languages and how they related to each other… Evolution of Computer Languages, here in diagram form Evolution Image