You might as I have asked yourself many times – What is inside NTDS.dit? (Most experienced Active Directory admins knows that NTDS.dit is the database and the physical on disk store that Active Directory uses to store information – most of you have probably got in touch with NTDS.dit during backup and restore scenarios)

Long story in a short version – I wasn’t satisfy not knowing – neither was I after being reading the following article:
(That I actually think isn’t that bad – but is also probably the most detailed public available information on the subject)
[1] http://technet.microsoft.com/en-us/library/cc772829(WS.10).aspx

So I decided with a very good friend of mine Christoffer Andersson (long-time Microsoft MVP in AD) to go ahead and build a tool that could read NTDS.dit and decode its internals, and then we started a journey that has given us invaluable knowledge at this part of Active Directory, this is the first article in a series of articles that will describe what’s really inside NTDS.dit and how Active Directory works on the database layer.

The illustration below has been presented in various documentations since Active Directory was initially released over 10 years ago; a similar illustration is also available in (However after this research project it’s actually turning out to be inaccurate in some aspects – in the way the DRA/REPL communicates with the DBLayer) [1]

Table 1: DSA Components (Simplified for the DBLayer)

Component

Description

Ntdsa.dll – Directory System Agent

The DSA, which runs as Ntdsa.dll on each domain controller, provides the interfaces through which directory clients and other directory servers gain access to the directory database (the DBLayer). In addition, the DSA enforces directory semantics, maintains the schema, guarantees object identity, and enforces data types on attributes.

Esent.dll – Extensible Storage Engine (ESE) APIs

The Extensible Storage Engine (ESE) is an advanced indexed and sequential access method (ISAM) storage technology. ESE enables applications to store and retrieve data from tables using indexed or sequential cursor navigation. It supports denormalized schemas including wide tables with numerous sparse columns, multi-valued columns, and sparse and rich indexes. It enables applications to enjoy a consistent data state using transacted data update and retrieval.

ESE was formerly known as Joint Engine Technology (JET) Blue, The DBLayer uses the ESE APIs documented here: http://msdn.microsoft.com/en-us/library/windows/desktop/gg269259(v=exchg.10).aspx

NTDS.dit

The on physical-disk file that represent the ESE/JetBlue database that holds the information store for the given DSA/Active Directory Domain Controller.

Data Store Physical Structure / Inside NTDS.dit - Tables

Finally we can start looking into the content/internal structure of NTDS.dit – but first let’s take a look on what has been reveled before, the illustration below is from [1] and is accurate as far as outside the white box that represent the tables within the database, the tables do exist (Except for * “sd_table” on Windows 2000 DSAs) – but there is more tables that isn’t mentioned in this example.

So it’s about time to reveal the real table structure of an NTDS.dit database file – It’s time to use the tool we produced to first discover this:

Table 2: NTDS.DIT - Tables

Table

Description

Minimum DSA Version

Datatable

Contains all objects and phantoms [2.1] represented as rows (1 object/phantom = 1 row in the table) from any instanced naming context (NC) held as either writable or read-only by the Directory System Agent (DSA) hosting the database and where columns represent every [2:3] attribute present in the schema except linked attributes [2:2]

[2.1]: phantoms are references to object’s hosted outside the given database (NTDS.DIT) and the given Directory System Agent (DSA)

[2:2] Post-Windows Server 2003 the attribute “ntSecurityDescriptor” is stored in the “sd_table” rather than in the “datatable”

[2:3] Some columns doesn’t reflect attributes and are columns pre-defined in the NTDS.dit template database generated by Microsoft (those are needed for internal states to the DSA)

Windows 2000 Server

Note: Windows Server 2008 R2 added a column to support the “is-Recycled” state

Hiddentable

Contains one row but several columns that defines the state of the database as well the [2:2] DNT (reference) of the NTDSA-Settings object that represents this DSA (used for finding config information specific to this domain controller.)

[2:4] The concept of DNTs (Distinguished Name Tags)

Windows 2000 Server
Note: Windows Server 2003 Introduced additional state columns such as backupexpiration_col

Link_table

Contains link-pair references (DNT, DNT), the link base (link id >> 1) and possibly a binary blob (In case of DN-binary, DN-string syntax)

Windows 2000 Server

Note: Windows Server 2008 R2 added a column to support deactivated links for recycle-bin

Sd_table

Contains single-instance-stored SDs (Security Descriptors) that pre-Windows Server 2003 was stored in the ntSecurityDescriptor attribute in the “datatable” – those are now instead referenced to the SDs in the “sd_table” that is, if more than one object has exactly the same security defined (Security Descriptor) both objects are referenced to the same row in the “sd_table”, hence the single-instance-storage and reducing the size needed to store Security Descriptors.

Windows Server 2003.

Sdpropcounttable

Used by the Security Descriptor Propagation Demon (SDProp) responsible for Security Descriptor inheritance down the tree, within the local database

 

Sdproptable

Used by the Security Descriptor Propagation Demon (SDProp) responsible for Security Descriptor inheritance down the tree, within the local database

Windows 2000 Server

Quota_rebuild_progress_table

Contains temporary information during quota tracking rebuild, for the Active Directory quota feature introduced in Windows Server 2003 – this allows the demon to keep track of processed objects.

Windows Server 2003

Quota_table

Contains quota tracking information, for the Active Directory quota feature introduced in Windows Server 2003, quota tracking is peer naming context (NC) and for a given security principal identified by its SID.

Windows Server 2003

MSysObjects

ESE Internals – out of scope for this article

N/A

MSysObjectsShadow

ESE Internals – out of scope for this article

N/A

MSysUnicodeFixupVer2

ESE Internals – out of scope for this article

N/A

In the next article – we will take a deep-dive into the content and the structure of the “datatable” also known as the object-store.


2 Responses to “How the Active Directory – Data Store Really Works (Inside NTDS.dit) – Part 1”  

  1. 1 satheesh

    Thank you so much for your great effort!!!!!

  2. 2 Vaibhav Sarmalkar

    Fantastic Work!!! Great Job…A detailed guide…this is exactly what i was looking for…
    Thanks a lot for all your efforts…

Leave a Reply