You might as I have asked yourself many times – What is inside NTDS.dit? (Most experienced Active Directory admins knows that NTDS.dit is the database and the physical on disk store that Active Directory uses to store information – most of you have probably got in touch with NTDS.dit during backup and restore scenarios)

Long story in a short version – I wasn’t satisfy not knowing – neither was I after being reading the following article:
(That I actually think isn’t that bad – but is also probably the most detailed public available information on the subject)
[1] http://technet.microsoft.com/en-us/library/cc772829(WS.10).aspx

So I decided with a very good friend of mine Christoffer Andersson (long-time Microsoft MVP in AD) to go ahead and build a tool that could read NTDS.dit and decode its internals, and then we started a journey that has given us invaluable knowledge at this part of Active Directory, this is the first article in a series of articles that will describe what’s really inside NTDS.dit and how Active Directory works on the database layer.

The illustration below has been presented in various documentations since Active Directory was initially released over 10 years ago; a similar illustration is also available in (However after this research project it’s actually turning out to be inaccurate in some aspects – in the way the DRA/REPL communicates with the DBLayer) [1]

Table 1: DSA Components (Simplified for the DBLayer)

Component

Description

Ntdsa.dll – Directory System Agent

The DSA, which runs as Ntdsa.dll on each domain controller, provides the interfaces through which directory clients and other directory servers gain access to the directory database (the DBLayer). In addition, the DSA enforces directory semantics, maintains the schema, guarantees object identity, and enforces data types on attributes.

Esent.dll – Extensible Storage Engine (ESE) APIs

The Extensible Storage Engine (ESE) is an advanced indexed and sequential access method (ISAM) storage technology. ESE enables applications to store and retrieve data from tables using indexed or sequential cursor navigation. It supports denormalized schemas including wide tables with numerous sparse columns, multi-valued columns, and sparse and rich indexes. It enables applications to enjoy a consistent data state using transacted data update and retrieval.

ESE was formerly known as Joint Engine Technology (JET) Blue, The DBLayer uses the ESE APIs documented here: http://msdn.microsoft.com/en-us/library/windows/desktop/gg269259(v=exchg.10).aspx

NTDS.dit

The on physical-disk file that represent the ESE/JetBlue database that holds the information store for the given DSA/Active Directory Domain Controller.

Data Store Physical Structure / Inside NTDS.dit - Tables

Finally we can start looking into the content/internal structure of NTDS.dit – but first let’s take a look on what has been reveled before, the illustration below is from [1] and is accurate as far as outside the white box that represent the tables within the database, the tables do exist (Except for * “sd_table” on Windows 2000 DSAs) – but there is more tables that isn’t mentioned in this example.

So it’s about time to reveal the real table structure of an NTDS.dit database file – It’s time to use the tool we produced to first discover this:

Table 2: NTDS.DIT - Tables

Table

Description

Minimum DSA Version

Datatable

Contains all objects and phantoms [2.1] represented as rows (1 object/phantom = 1 row in the table) from any instanced naming context (NC) held as either writable or read-only by the Directory System Agent (DSA) hosting the database and where columns represent every [2:3] attribute present in the schema except linked attributes [2:2]

[2.1]: phantoms are references to object’s hosted outside the given database (NTDS.DIT) and the given Directory System Agent (DSA)

[2:2] Post-Windows Server 2003 the attribute “ntSecurityDescriptor” is stored in the “sd_table” rather than in the “datatable”

[2:3] Some columns doesn’t reflect attributes and are columns pre-defined in the NTDS.dit template database generated by Microsoft (those are needed for internal states to the DSA)

Windows 2000 Server

Note: Windows Server 2008 R2 added a column to support the “is-Recycled” state

Hiddentable

Contains one row but several columns that defines the state of the database as well the [2:2] DNT (reference) of the NTDSA-Settings object that represents this DSA (used for finding config information specific to this domain controller.)

[2:4] The concept of DNTs (Distinguished Name Tags)

Windows 2000 Server
Note: Windows Server 2003 Introduced additional state columns such as backupexpiration_col

Link_table

Contains link-pair references (DNT, DNT), the link base (link id >> 1) and possibly a binary blob (In case of DN-binary, DN-string syntax)

Windows 2000 Server

Note: Windows Server 2008 R2 added a column to support deactivated links for recycle-bin

Sd_table

Contains single-instance-stored SDs (Security Descriptors) that pre-Windows Server 2003 was stored in the ntSecurityDescriptor attribute in the “datatable” – those are now instead referenced to the SDs in the “sd_table” that is, if more than one object has exactly the same security defined (Security Descriptor) both objects are referenced to the same row in the “sd_table”, hence the single-instance-storage and reducing the size needed to store Security Descriptors.

Windows Server 2003.

Sdpropcounttable

Used by the Security Descriptor Propagation Demon (SDProp) responsible for Security Descriptor inheritance down the tree, within the local database

 

Sdproptable

Used by the Security Descriptor Propagation Demon (SDProp) responsible for Security Descriptor inheritance down the tree, within the local database

Windows 2000 Server

Quota_rebuild_progress_table

Contains temporary information during quota tracking rebuild, for the Active Directory quota feature introduced in Windows Server 2003 – this allows the demon to keep track of processed objects.

Windows Server 2003

Quota_table

Contains quota tracking information, for the Active Directory quota feature introduced in Windows Server 2003, quota tracking is peer naming context (NC) and for a given security principal identified by its SID.

Windows Server 2003

MSysObjects

ESE Internals – out of scope for this article

N/A

MSysObjectsShadow

ESE Internals – out of scope for this article

N/A

MSysUnicodeFixupVer2

ESE Internals – out of scope for this article

N/A

In the next article – we will take a deep-dive into the content and the structure of the “datatable” also known as the object-store.

A fellow Microsoft MVP has assembled a large list (> 6MB) of domain names/IP addresses mainly used to serve advertisements, tracking cookies, and sometimes even malware.

You can find the full instructions  at http://www.mvps.org/winhelp2002/hosts.htm. I personally have been using this list for almost two years now (it is frequently updated).

Because I have IIS set-up on my main machine I decided to change the redirect IP address to 0.0.0.0 instead of 127.0.0.1, so that IIS is not “bothered” with the failed requests.

P.S. This list also blocks the YouTube commercials, which is a nice touch!

What are threads?

A thread executes code synchronously—a set of instructions processed by the CPU “first in first out”. This can include virtually any kind of code, be it updating of the GUI, processing calculations, or waiting for user input.

Windows Forms is arguably one of the most used aspects of .NET Framework. GUI programs typically rely on two layers:

  • Processing stack, which includes any code or task executed in the background, and
  • User interface, which is supposed to give the user a means for instructing the program to execute tasks, and receiving timely “updates” or indications of progress.

Multithreading

Multithreading is a programming paradigm, which has become popular during the past decade. In a nutshell, the CPU can switch between two or more thread contexts so two tasks (e.g. calculation, and GUI update) can be executed in parallel—or asynchronously.

For example, a program can make full use of a multi-core processor by instructing each core to process a separate task, or a chunk of the whole task. This results in improved performance.

In the case with Windows Forms, a separate thread can be used to process a task while allowing the main thread (GUI) to provide timely information about the progress, or to let the user continue their work during the lengthy task.

Continue reading ‘Multithreading with Windows Forms in C#’

Честита Коледа, и приятно посрещане на Новата година на всички читатели на блога!

Също така поздравления на Miguel Hernandez (@migue333) и Kim Nilsson (@no_substitute), които бяха избрани посредством случаен жребий от Bryant Zadegan и мен, за да получат по един подарък-карта за абонамент MSDN Ultimate.

Честито!Благодарности на Microsoft за подкрепата.

Previous readers of my blog might already know that I am a big fan of ImageX, a tool that has been supplied by Microsoft in the Windows Automated Installation Kit since the release of Windows Vista.

A quick introduction

ImageX is a command-line tool that enables original equipment manufacturers (OEMs) and corporations to capture, to modify, and to apply file-based disk images (.wim files) for rapid deployment.

WIM files can capture disk or directory snapshots in such a that identical files are only stored once in the internal structure of the image. In addition to this, compression can be used to minimize the size of the final data.

These are some of the reasons I have been using ImageX to take full “snapshots” of all of my code, private files, etc. from time to time, for backup  and archiving purposes. When needed, I can extract different revisions of the same file, or restore a whole image/snapshot in case of disk damage or data loss.

wim_graphThe graph on the left visualizes the changes to my projects backup file spread across 16 months. The last image’s actual data weighs at 64GB, while the whole WIM is 59GB—and it contains all file changes, directory structures, even information that has long been deleted from my working copy.

Requirements

  • The scripts package. Download and extract to a folder of your choice.
  • ImageX.
    1. Download and install the Windows Automated Installation Kit (WAIK).
    2. Copy the ImageX binaries (both amd64 and x86 folders), located in \Program files\Windows AIK\Tools\ to the backup\app\imagex folder.

Continue reading ‘A simple backup solution using ImageX, a Windows Imaging tool’

SDC14015 - Resized

Just wanted to post a quick update:

Thanks to the MVP Program I have received two MSDN Ultimate Subscription gift cards, worth $11,899 each.

These are going to be given away soon as I am working with Bryant on the giveaway details. Stay tuned.

FlacReader + WavWriter

As a follow-up to my previous article on FLAC and encoding uncompressed audio, I have further developed the Wav2Flac library and added FLAC decoding to the WAVE container. Some key points of this aspect of the WavFlacTest library are:

  • Support for 16- and 24-bit audio streams
  • Support for virtually all FLAC channel mappings: mono, stereo, 5.1, 7.1, etc.
  • The resulting file is a bit-for-bit copy of the original audio stream

All source code and the compiled 32-bit FLAC library can be downloaded from here.

Known limitations

  • Due to the wave format used being 32-bit, uncompressed files greater than 2GB in size might not play in full in some players.
  • Even though FlacReader can receive metadata callbacks, the data is not interpreted. This can be further developed using the FLAC API documentation.

Inline Tweet Translator

Translated tweet from @katyperryOne thing that popped in my mind yesterday after having a conversation over twitter (admittedly, a service I had rarely used until recently), is that the more followers you have, the higher the odds you would have to use other languages in addition to a lingua franca. This ‘leaves out’ other friends who could have limited knowledge of foreign languages.

This is how the idea for the Tweet Balloon Translator was born. It’s a Greasemonkey script for Firefox, which uses the balloon/hover feature recently introduced in twitter plus the Google Translate service to translate tweets quickly and inline.

You can install the script from userscripts.org or the local copy here.

Development notes

The key phases I outlined for the script were as follows:

  1. Establish a working translation block of code using the Google Translate JSON API
  2. Research the possibilities of inserting a Translate link in such aspects of twitter as the profile page, direct messages, retweets. The Guest/logged out user scenario also had to be taken into account as twitter serves the pages in a different manner.
  3. Figure out a way to use the twitter internal API for the HoverCard balloon feature. This was probably the hardest of all three because of the way these are created and populated.
  4. Integrate 1 through 3 in a single script.

Needless to say, the script employs asynchronous requests to Google via the HTTPXMLRequest model so as to not break the UI thread, as well as exception handling.

Please feel free to try out, review, or contribute to the script at userscripts.org. I feel both international and English-speaking users could benefit from this by being able to communicate with more people on twitter, and understand international tweets.

Resizing a form while keeping aspect ratio is useful in many cases, like video playback or vector graphics. This way, the window can be resized while retaining the original ratio and avoiding the use of letterboxing or pillarboxing.

What’s needed is for the window function to be overriden (WndProc) and pre-process the target window rectangle used by the WM_SIZING message.

The new destination rectangle is calculated by taking into account the resizing handle and the window chrome size (title height, border width, etc.).

protected override void WndProc(ref Message m)
{
    if (m.Msg == WM_SIZING)
    {
        RECT rc = (RECT)Marshal.PtrToStructure(m.LParam, typeof(RECT));

        int w = rc.Right - rc.Left - chromeWidth;
        int h = rc.Bottom - rc.Top - chromeHeight;

        switch (m.WParam.ToInt32()) // Resize handle
        {

           // …

        }

        Marshal.StructureToPtr(rc, m.LParam, true);
    }

    base.WndProc(ref m);
}

You can find the full C# source code here, including a test program. The aspect ratio and initial client size is set to 16:9.

Why lossless?

Lossless audio is used on various media, including studio masters, CD, DVD-Audio (via MLP) and Blu-ray (via Dolby TrueHD, which is technically a rebrand of and an extension to MLP, and DTS-HD Master Audio). All of these, when decoded, will result in a pulse-code modulated signal identical to the source, unlike the popular MP3 format. MP3 performs a quality-file size trade-off by discarding or reducing frequencies less audible to human hearing.

PCM by design uses a constant bitrate, which is proportional to the sample rate, bit depth and number of audio channels, which results in very large file sizes with the increasing of each parameter, and/or duration of the audio track.

Solutions such as FLAC, TrueHD and DTS-HD MA are used to losslessly compress the source audio so that the rest of the medium (for example, a Blu-ray disc) can be used for more audio tracks, higher-bandwidth video or extras.

Out of the aforementioned, only FLAC is free to use—both TrueHD and DTS-HD MA encoders and decoders have to be licensed.

The first part of the series will explore the processing of uncompressed audio data with the FLAC API in C#. In case you prefer to use Visual Basic .NET, you can use this online converter.

Continue reading ‘Encoding uncompressed audio with FLAC in C#’