The K Blag » replication

Determine CSN update time from CSR in Directory Server 5/6

Mike — Mon, 22 Dec 2008 16:50:38 +0000

I needed to determine the update time for a particular attribute for one of my customers. I knew I had ran across how to get it from a csn attribute.

If you can obtain the CSN of an attribute, ie: from the nscpentrywsi operational attribute), the timestamp is the first 8 bytes of the CSN, in hex format.

ie:

nscpentrywsi: cn;adcsn-494d685d000000010000;vucsn-494d685d000000010000: The K

The timestamp in hex is: 494d685d

With a bit of perl magic, we can obtain the localtime value:

perl -e ‘print localtime(hex(”494d685d”)) . “\n”;’

Which returns:

Sat Dec 20 16:49:17 2008

bookmark to

Backing up and Restoring data from Directory Server 6

Mike — Mon, 26 Nov 2007 19:54:10 +0000

One of the largest problems that I have noticed when doing directory deployments, are either invalid, or non-existent backups of their directory information. They often assure peace of mind when things unexpectedly go wrong (a la unexpected power outages).

I’ve found a good reference to how to do backups with Directory Server 6; and won’t rehash them here.

In essence, it may become necessary to have a single instance dedicated to doing routine backups in a large environment. This is because you need to be able to turn the instance to read-only for the duration of the backup.

bookmark to

Sun Directory Server 5.2 6-Way MMR – part 8 – Enable Directory Server backends for updates

Mike — Tue, 20 Nov 2007 15:00:24 +0000

Once the directory server replica’s are configured and initialized, the final step is to enable them to accept updates.

This step was added as there is a necessary catch-up time when initializing MMR (especially in a busy environment); where by the time an instance is initialized, it is out-of-date until all the subsequent updates have been propagated. When I had to do this the first time, I was working off static data, and didn’t need to worry about performing this when the updates have been propagated, as there were no updates.

The only instance and suffixes that do not need to be enabled are those on ultimate master, which had all the data initially. All other instances need to be enabled after the replica’s have caught up.

In summary, those instances are:

ds0: nothing needs to be enabled

ds1: user/group, o=internet, o=piserverdb, o=pab, o=comms-config, o=netscaperoot

ds2: user/group, o=internet, o=piserverdb, o=pab, o=comms-config, o=netscaperoot

ds3: user/group, o=internet, o=piserverdb, o=pab, o=comms-config, o=netscaperoot

ds4: user/group, o=internet, o=piserverdb, o=pab, o=comms-config, o=netscaperoot

ds5: user/group, o=internet, o=piserverdb, o=pab, o=comms-config, o=netscaperoot

I’ve created an example command set to enable a directory server backend.

bookmark to

Enable Directory Server 5.2 database back-end

Mike — Mon, 19 Nov 2007 15:43:32 +0000

In a MMR agreement, the consumer instance backend that has been initialized needs to be enabled before it can begin accepting replication updates. This is a feature to allow a newly replicated master to catch-up on changes that have occurred during the initialization.

An example LDIF of how to enable a backend:

dn: cn=replica, cn=“suffix“,cn=mapping tree,cn=config
changetype: modify
add: ds5BeginReplicaAcceptUpdates
ds5BeginReplicaAcceptUpdates: start

I created and saved the LDIF to enable this backend suffix as /tmp/enablesuffix.ldif; and ran the ldapmodify command to perform the enable.

/usr/bin/ldapmodify -h host -p port -D ‘rootdn‘ -w ‘password‘ -c -f /tmp/enablesuffix.ldif

More information about MMR convergence from Sun.

bookmark to

Sun Directory Server 5.2 6-Way MMR – part 7 – Initialize replication agreements

Mike — Fri, 16 Nov 2007 15:03:03 +0000

To be usable in an MMR, all the suffixes on each instance that are part of the MMR (or replication in general) need to be initialized from the ultimate master (where the data was initially loaded). As some of the instances are not directly replicated from this instance (as there are a maximum of 3 MMR agreements from this instance); but instances which do replicate to them can initialize that data.

In order to successfully initialize all the replica’s in a short amount of time, the first master ( where all the initial data was loaded ), first replicates to a counterpart; then they both replicate to another instance that had not been replicated before. With alot of data, this can take hours to perform.

One methodology I’ve used is:

ds0 initializes all suffixes on ds3
ds0 initializes all suffixes on ds1 & ds3 initializes all suffixes on ds4
ds0 initializes all suffixes on ds5 & ds3 initializes all suffixes on ds2

Do so in this manner will allow you to initialize all the instance suffices in a relatively short amount of time.

After initializing a suffix, before it can accept updates, it needs to have its database back-end enabled, which I’ll write about at a different time.

I’ve created a short example of how to initialize a replica.

bookmark to

Initialize a database suffix in Directory Server 5.2

Mike — Thu, 15 Nov 2007 15:13:11 +0000

In a replicated environment, all (initial) data should flow from a single master to all other instances. After preparing the (destination) directory server for replication, the remote instance database needs to be initialized.

An example replication initialization:

dn: cn=replication agreement name:389,cn=replica,cn=suffix,cn=mapping tree,cn=config
changetype: modify
replace: nsds5BeginReplicaRefresh
nsds5BeginReplicaRefresh: start

With our LDIF written as /tmp/reinit.ldif; run the following ldapmodify command:

/usr/bin/ldapmodify -h host -p port -D ‘rootdn‘ -w ‘password‘ -c -f /tmp/reinit.ldif

bookmark to

Sun Directory Server 5.2 6-Way MMR – part 6 – Create replication agreements

Mike — Wed, 14 Nov 2007 14:42:03 +0000

Up until now, we’ve been preparing each of our instances to participate in MMR; at this point, we create the replication topology and the replication agreements.

Being as each of our 6 instances will be masters, and with Directory Server 5.2, there is a limit of 4 masters in an MMR, a bit of creativity must be employed. Each of the master instances will replicate each of its back-end databases (user/group, o=internet, o=pab, o=piserverdb, o=comms-config and o=netscaperoot) to each of its immediate peer instances, meaning those that are numbered sequentially lower and higher; as well as another instance that is neither sequentially lower or higher.

The replication agreement list is defined below.

ds0 -> ds5 / ds1 / ds3

ds1 -> ds0 / ds2 / ds4

ds2 -> ds1 / ds3 / ds5

ds3 -> ds2 / ds4 / ds0

ds4 -> ds3 / ds5 / ds1

ds5 -> ds4 / ds0 / ds2

So, the instances on the instance ds0 will replicate each of its back-end databases to ds5, ds1 and ds3. Being as each master has only 3 MMR replication agreements per backend, they each believe they are in a 4-way MMR; while in actuality, there are 6 masters serving the data.

For information on how to create the actual replication agreement LDIF; please follow my instructions.

bookmark to

Create Directory Server 5.2 replication agreements

Mike — Tue, 13 Nov 2007 15:32:39 +0000

In a Sun Directory Server 5.2 multi-master replication (MMR) topology, the instances that are to master the data, ie: accept changes and push those changes to other instances; need to have a replication agreement per-replicated-suffix / per-replicated-instance.

To create a replication agreement, the instance being replicated to needs to be configured to accept replicated changes. We create this LDIF with the full contents of a replication agreement for a Directory Server 5.2 instance. Replace all italicized items with the appropriate values for your deployment.

dn: cn=ds1.example.com:389, cn=replica, cn=”dc=example,dc=com“,cn=mapping tree,cn=config
changetype: add
objectClass: nsDS5ReplicationAgreement
objectClass: top
nsDS5ReplicaHost: ds1.example.com
nsDS5ReplicaPort: 389
nsDS5ReplicaBindMethod: SIMPLE
nsDS5ReplicaBindDN: cn=replication manager, cn=replication, cn=config
nsDS5ReplicaUpdateSchedule: *
description: user and group data to ds1:389
nsDS5ReplicaRoot: dc=example,dc=com
cn: ds1.example.com:389
nsDS5ReplicaCredentials: remotereplmanpwd

I created and saved my replication agreement as /tmp/replagmt.ldif; and ran the ldapmodify command to create the above replication agreement.

/usr/bin/ldapmodify -h host -p port -D ‘rootdn‘ -w ‘password‘ -c -f /tmp/replagmt.ldif

bookmark to

Sun Directory Server 5.2 6-Way MMR – part 5 – Enabling replication on a back-end

Mike — Mon, 12 Nov 2007 16:11:17 +0000

In order to participate in replication, each of the suffixes that are to be replicated need to be configured for replication. Each MMR suffix should have a unique replication identifier associated with it, numbered 0 to 65536.

With our choice of applications and associated schema, we have 6 unique suffixes, each with their own purpose:

dc=example,dc=com

The primary user/group suffix. We used the numbers 10000 to 10005 to identify MMR instances.

ds0 / dc=example,dc=com : 10000

ds1 / dc=example,dc=com : 10001

ds2 / dc=example,dc=com : 10002

ds3 / dc=example,dc=com : 10003

ds4 / dc=example,dc=com : 10004

ds5 / dc=example,dc=com : 10005

o=comms-config

The location of the Communications configuration information. We used the numbers 20000 to 20005 to identify MMR instances of this type.

ds0 / o=comms-config : 20000

ds1 / o=comms-config : 20001

ds2 / o=comms-config : 20002

ds3 / o=comms-config : 20003

ds4 / o=comms-config : 20004

ds5 / o=comms-config : 20005

o=internet

Holds the schema1 mail routing information for local domains. We used the numbers 30000 to 30005 to identify MMR instances of this type.

ds0 / o=internet : 30000

ds1 / o=internet : 30001

ds2 / o=internet : 30002

ds3 / o=internet : 30003

ds4 / o=internet : 30004

ds5 / o=internet : 30005

o=NetscapeRoot

Holds primarily console information, if all instances are configured in the same instance, all instances can show up in the console. We used the numbers 40000 to 40005 to identify MMR instances of this type.

ds0 / o=NetscapeRoot : 40000

ds1 / o=NetscapeRoot : 40001

ds2 / o=NetscapeRoot : 40002

ds3 / o=NetscapeRoot : 40003

ds4 / o=NetscapeRoot : 40004

ds5 / o=NetscapeRoot : 40005

o=pab

Sun Messaging Personal Address Book. We used the numbers 50000 to 50005 to identify MMR instances of this type.

ds0 / o=pab : 50000

ds1 / o=pab : 50001

ds2 / o=pab : 50002

ds3 / o=pab : 50003

ds4 / o=pab : 50004

ds5 / o=pab : 50005

o=PiServerDB

Sun Unified Web Client Address Book. We used the numbers 60000 to 60005 to identify MMR instances of this type.

ds0 / o=PiServerDB : 60000

ds1 / o=PiServerDB : 60001

ds2 / o=PiServerDB : 60002

ds3 / o=PiServerDB : 60003

ds4 / o=PiServerDB : 60004

ds5 / o=PiServerDB : 60005

To enable replication, for each of the hostname and suffix combinations, please follow the instructions I have created.

bookmark to

Preparing Sun LDAP Directory Server 5.2 Suffix for Replication

Mike — Sun, 11 Nov 2007 17:06:34 +0000

In order for a directory server instance to participate in multi-master replication, each suffix that is to participate in MMR needs to be configured with a unique (at the very least among MMR for the suffix) integer that identifies this master in MMR. The numbers can range from 0 to 65535 to identify these MMR instances.

To identify the type of replica this instance should be, we need to address the nsDS5ReplicaType attribute. A value of 2 means this instance will be a consumer instance only, a value of 3 means this instance will be a read/write (master) instance. Values of 0 and 1 should not be used. More information about the nsDS5ReplicaType attribute.

dn: cn=replica, cn=”suffixname“,cn=mapping tree,cn=config
changetype: add
objectClass: nsDS5Replica
objectClass: top
cn: replica
nsDS5ReplicaBindDN: cn=replication manager, cn=replication, cn=config
nsDS5ReplicaRoot: suffixname
nsDS5Flags: 1
nsDS5ReplicaType: 3
nsDS5ReplicaId: uniqueidentifier

Create a file with the above contents and save it to disk (I saved mine as /tmp/enablereplsuffix.ldif).

Update your directory server instance via ldapmodify to enable replication to this suffix on this host.

/usr/bin/ldapmodify -h host -p port -D ‘rootdn‘ -w ‘password‘ -c -f /tmp/enablereplsuffix.ldif

bookmark to