Directory Sync Revisited: You Get What You Pay For

by Jan Trumbo and Joel Snyder

Electronic mail systems are being joined to an enormous networked mesh, stretching not just across the corporation, but the continent. Integrating very different kinds of e-mail has become a daily challenge for many network managers.

One of the most difficult parts of linking e-mail systems is keeping their directories in sync. As a tide of users ebbs and flows through different e-mail systems, a unified directory has become an elusive target. Without directory synchronization, mail may be harbored in abandoned corners of the network or, even worse, disappear into the ether.

Some organizations have found the problem of building a single e-mail directory so difficult that they've chosen to go without it, using "blind addressing." In blind addressing, messages sent from one e-mail system to another don't use the directory--they require the sender to know the full e-mail address of the recipient. Therein lies the problem: Network users want friendly e-mail systems with built-in directories that keep the entire company at their fingertips.

Last year, Network Computing tested four directory synchronization products based on OpenVMS (see "Directory Assistance,"). For this round, we invited all of the Unix-based directory sync and mail backbone vendors to show us their stuff. We found that it's easier for them to talk the talk than to walk the walk: Only three vendor were willing to sit down and show us their stuff.

Control Data Systems' Mail*Hub 96 product, Worldtalk Corp.'s Worldtalk and Hitachi Computer Products' SyncWare (formerly called Mosaic) all offered themselves up for scrutiny. We found that you get what you pay for--and the more money you have to throw at the problem, the better solution you'll get.

Mail*Hub, the premium mail backbone and hub system, had the best coverage, excellent manageability and a fantastic feature set for directory synchronization. It also costs twice as much as the next cheapest package and requires extensive on-site customization to make it perfect.

The two less expensive products, Worldtalk and SyncWare, both handle directory synchronization competently but with less grace and management flexibility. Worldtalk takes a centrist approach, with all configuration and control coming from the mail backbone server. Hitachi chose a more distributed view, pushing management out to the individual LAN e-mail systems.

Looking at Directories We wanted to see how each system handled synchronizing the directories of different e-mail systems. Most of the time was spent on PC LAN e-mail systems, such as Microsoft Mail for PC Networks (MS Mail) and cc:Mail. We also investigated how these packages support legacy e-mail systems, such as Digital's All-in-One and several different flavors of IBM's PROFS and OfficeVision--honors went to CDS. We also created a few problem cases to see how each system responded to real-world challenges. These are described in "The Hard Stuff," on page 104.

In all cases, we ran the tests with the vendor on-site. For CDS, our testers went to the CDS labs in Minneapolis because the company felt that installation time of its product in our environment would make real-world testing prohibitive. Worldtalk and Hitachi both sent experienced engineers to our labs to help install the software in our existing network.

Control Data Systems' Mail*Hub 96

Overall, we found that CDS Mail*Hub was the best engineered and best supported product of the three. Unfortunately, Mail*Hub is priced out of the range of most smaller networks, which means that it cannot be everyone's first choice. However, for very large e-mail networks, Mail*Hub must be a candidate for both backbone services and directory synchronization.

CDS' Mail*Hub is a set of integrated products, including backbone mail connectivity (based on X.400) and PC LAN mail gateways. The products are aimed at message transfer, with add-ons for directory synchronization, document conversion and a spiffy X-based GUI management interface.

Mail*Hub runs on Solaris (SPARC or Intel), HP/UX and IBM/AIX systems. We tested it on an HP 9000 running HP/UX. Mail*Hub directory synchronization is tied intimately to Mail*Hub's message transport service--you can't have directory synchronization without message transport. Control Data Systems is an integrator: When you buy Mail*Hub, you typically also buy weeks or months of CDS consulting time to engineer it, install it, make it work and even manage it. In fact, you don't have any choice. CDS doesn't sell parts. It sells service. If you have a CDS-based backbone, CDS installed it.

Mail*Hub is geared for big-time operations--CDS won't even talk to you unless you have at least 5,000 users, but 10,000 is a more typical number. Entry-level CDS installations usually begin at $100,000 and go up from there, so cost is a major consideration. CDS is for big e-mail systems, and it does big e-mail very, very well.

Mail*Hub's directory synchronization uses a central X.500 database built into the Mail*Hub product. An X.500 database has a strong design and it is extensible without breaking the directory sync.

To understand how CDS works, let's look at the example of a Novell GroupWise domain, which might have multiple post offices behind it. CDS provides software components (agents) that run on the GroupWise messaging server. These agents use the GroupWise directory import and export interfaces to pull down changes (deltas) to the GroupWise directory.

Directory synchronization is driven by a process in the mail hub where directory updates and reconciliations occur. It periodically polls all the agents (including our example, GroupWise) and asks them to generate these deltas and send them back.

When our example agent gets a poll, it asks GroupWise for a delta, and sends the information it pulled from the directory to the CDS hub. When the directory delta information shows up at the hub, the directory synchronization process that requested the delta goes to work. It reads the delta information and uses that to update the master X.500 directory.

After all of the e-mail systems have submitted their deltas and the entire X.500 database has been updated with the changes, the directory synchronization process moves into the next stage: generating deltas to send out to the agents at each of the synchronizing e-mail systems. In our example, the CDS GroupWise agent eventually gets its own delta from the mail hub, which it must apply to the GroupWise directory. This delta has all of the names and mail addresses that have changed on all of the other e-mail systems that the CDS hub is synchronizing. Once the process is finished, the mail administrator receives an e-mail message summarizing the changes.

After looking at the basic directory synchronization and special problem cases, we decided that it was a lot easier to handle directory collisions in the PC e-mail systems than in the Mail*Hub. If you can't do that, for whatever reason, CDS gives you sufficient power to do whatever you want. There are user callouts to do almost anything you want, and if we wanted to do something that wasn't covered, the source code for the CDS directory synchronization software is provided.

Management of the directory synchronization is great. Network managers can actually manage: hide people, cause them to be updated or even deleted from the PC LAN systems, change preferred mail address and adjust the synchronization schedule. All of this is done through an easy-to-use X Window-based GUI. On top of that, the e-mail users can use X.500 access tools, including Web browsers, to adjust their own information, subject to permissions granted by the mail manager. The breadth and depth of interfaces and options available to the network manager and the e-mail user far exceed that which is available in the other products.

External access to the Mail*Hub directory was also very extensive. Internet correspondents could use an X.500 browser, finger, ph, and whois a World Wide Web browser or query-by-mail to find CDS users in the X.500 directory. The X.500 directory is at the core of the CDS product: Every incoming Internet or X.400 message uses the X.500 directory to find a final destination.

Worldtalk Corp.'s Worldtalk

Like Mail*Hub, Worldtalk is a complete messaging system. Based on HP's X.400 Message Transfer Agent (MTA), Worldtalk is fundamentally a series of gateways and additions to an X.400 e-mail backbone. Worldtalk's directory synchronization software works in conjunction with its backbone. Worldtalk's backbone software runs on HP hardware.

Worldtalk is considerably simpler to install than Mail*Hub. The product comes with on-site installation, but after that, clients are free to configure and manage it themselves or purchase services from Worldtalk. In our case, Worldtalk helped us get the system up and running in less than two days, and working with two of our test PC LAN e-mail systems.

Overall, Worldtalk's e-mail backbone is more reasonably priced than the CDS backbone. The backbone is priced on a per-user basis, starting at about $30 per user for an installation with 1,000 users and going down from there. A starter installation for a few PC LAN e-mail systems and 1,000 users, including hardware, will set you back about $50,000.

Worldtalk's directory system stores all information in a Raima Corp. db_vista database on the central mail server. The database is designed just for directory synchronization, and access to it is provided only through Worldtalk-supplied applications.

The design of directory synchronization in Worldtalk is very centrist--an excellent approach. All smarts and configuration occurs on the central mail server. The Worldtalk-supplied software at the PC LAN e-mail systems collect and deposit directory information under orders from the central mail server.

Worldtalk's approach to synchronization is very similar to CDS, with one major difference: Each e-mail system is handled separately without coordination in timing between different e-mail systems. For example, we set up cc:Mail and MS Mail to synchronize across a Worldtalk mail server. Worldtalk's mail server first commands the cc:Mail access unit software (this is analogous to CDS' "agents") running on a PC to pull down a list of changes to the cc:Mail directory. This is sent to the central mail server where it is used to update the central directory. Then, Worldtalk generates a list of changes to the cc:Mail e-mail directory to bring it into synchronization with the central directory and ships those changes down to the cc:Mail access unit software. The access unit is responsible for importing those changes into the cc:Mail directory. Lastly, the Worldtalk mail server starts on MS Mail, asking for changes, reconciling them in the directory and shipping the update to MS Mail.

The important difference is the asynchronous nature of the synchronizations. Even if the cc:Mail and MS Mail had overlapped, each delta received from the access unit software generates an update that is sent immediately. Worldtalk does not attempt to collect all of the e-mail directories before generating a synchronization transaction. This is a very simple strategy that is easy to manage, but there is one drawback: Changes at one end of the e-mail network can take a long time to propagate back to the other end. If your network doesn't change very often, or if the delay is not a problem, this approach will probably work for you. Worldtalk reports that it is moving to a more global synchronization that would reduce the delay in propagating changes, similar to the approach used in Mail*Hub.

Worldtalk offers an X.500 gateway to its directory synchronization system, but you're still limited to the internal database schema--X.500 is really an access method to the existing directory, synchronizing itself like any other LAN e-mail system.

External access is very limited in the Worldtalk server: query-by-mail is the only direct user access to the central directory database. However, an installation that required online access to the database could use the X.500 product as a synchronized shadow of the central e-mail directory.

For directory synchronization, Worldtalk did a solid job of keeping e-mail systems together. With many of the features of Mail*Hub missing, Worldtalk also lacks CDS' premium price. For the money, Worldtalk does a very credible and reliable job.

Hitachi Computer Products' SyncWare

SyncWare is a directory synchronization package only. Unlike Mail*Hub and Worldtalk, it is not part of a larger mail backbone. For networks that already have a backbone in place, SyncWare layers on top of the existing transports to synchronize e-mail directories.

The work of reconciling directories occurs on the SyncWare server. We tested SyncWare on a Sun workstation running SunOS. The server software can also run on Solaris and HP/UX.

Hitachi provides Windows-based software for each of the client e-mail agents. These are responsible for running the client directory import and export programs. Herein lies a fundamental difference between Hitachi and the other packages: SyncWare is driven by the agent side, not the server side. Although it is possible for the server to force an agent to synchronize, the normal operation has agents making the decision of when and what to synchronize. The server has a much more subordinate role in Hitachi's view.

Hitachi's pricing model is user-based, with the server sized according to the maximum number of e-mail users. It is also the least expensive product, if all you want is synchronization, with costs typically less than the other packages we examined. For example, a 1,000-user system with two e-mail LANs to be synchronized would cost about $15,000, including hardware.

Hitachi's agents are hardware-intensive. The product basically requires a dedicated PC per LAN e-mail system. However, Worldtalk lets you synchronize multiple LAN e-mail systems with a single PC. Of course, PC hardware is hardly a significant expense in this type of system.

SyncWare covers the major e-mail systems, despite significant gaps, such as Digital's office automation products. It has a strong "do-it-yourself" feel and lets you build scripts with its "universal agent" to synchronize any e-mail system that generates ASCII-based directory information.

SyncWare uses an embedded database to store directory information with its own schema. No X.500 gateway is available and extensions to the vendor-supplied schema are not allowed.

To synchronize a directory, the network manager must start the agent software by scheduling it to run at specific intervals. When an agent wakes up, it extracts a delta from its assigned e-mail system and e-mails the data to the server with a request for an update.

The server receives the e-mail and uses it to update the central database. When the update and reconciliation is complete, it sends a confirmation back to the agent at the client end. The agent then sends another e-mail to the server requesting an update to its own e-mail directory. The server receives, complies and returns the requested data to the agent, which uses it to update the client e-mail directory with new information.

Because SyncWare is detached from a strict synchronization cycle, it works well over unreliable networks, such as sporadically connected branch offices in lesser-developed countries. SyncWare's agent orientation requires distributed configuration and control. All of the rules about converting system-specific information (such as the mapping of names and addresses to a canonical format) is handled on the agent end, not the server end. This puts a great deal of power in the hands of the client e-mail system administrator. Unfortunately, editing these files and creating the rules with Unix-based regular expressions is a difficult task. The GUI provided with SyncWare is nearly useless. We resorted to a simple text editor to change the configuration files.

We were not impressed by SyncWare's management capabilities. Error messages are worse than cryptic. For example, in our test configuration, an error message kept popping up in the logs telling us a file could not be found, but never telling us the directory it was searching. Server-side database management is also cumbersome, reflecting the agent-based orientation of the product. We found it difficult to find detailed information about users in the database with the GUI interface provided. SyncWare is not meant to be configured by mere humans. This was the least approachable management interface we tested.

SyncWare supports external access using query-by-mail. For SMTP users, SyncWare exports to a Unix-standard aliases file, which could also be searched using some site-specific protocol or software.

SyncWare's particular strength is in the anarchy of the system. Because Hitachi doesn't force you to buy a particular message transport and because the orientation of the product is toward LAN manager control, rather than central manager control, it would fit well into an environment where mutually warring network managers could not agree on a central management strategy.

Jan Trumbo is a postmaster at the University of Arizona's telecommunications department. She can be reached at Joel Snyder is a senior partner with Opus One, Tucson, Ariz., specializing in networks and international aspects of information technology. He can be reached at

The Hard Stuff: A Real-World Directory Synchronization Challenge

ll of the products will do what a network manager asks--sync directories of multiple e-mail systems. We were interested in how these products fared when "problem" cases arose. These are the inevitable name collisions and naming problems that are found in any real-world e-mail system.

We created four separate tests that we used to understand how each system works:

· Two different people with the same name.
· Two identical names that are the same person.
· Two different names that are the same person.
· People who don't want to be propagated to other e-mail systems.

In our first problem case, two different people with the same name, CDS gave us a bewildering array of choices. We chose to pick the most useful solution and had the CDS hub actually change the appearance of the two names in all e-mail systems so that users could tell them apart. We had one user show up as "John Bigboote (SALES)" and the other as "John Bigboote (ADMIN)." This had to be dealt with manually, although the CDS people told us that with sufficient PERL expertise, we could have made their product do it automatically.

Worldtalk's LAN-driven architecture didn't offer the same opportunities. Worldtalk identified the problem in a management report, but the only way to accomplish our goal was to go to the actual LAN e-mail systems and update them, which solved the problems automatically at the central mail server.

SyncWare's client-driven management also restricted our options. Handling problems has to be done at the client end with special rules or modifications to the existing LAN system. The server manager can't affect the behavior of the sync process. All the server does is reconcile deltas--there are no mapping rules. The solution to this problem was the same as Worldtalk: Go to the client and change it.

The second case, a user with mailboxes on multiple systems, was harder to solve with all products. Worldtalk and SyncWare have "first come, first served" policies--whichever e-mail system synchronizes first is the one that wins the place in the database. We got management reports that reported the problem, but solving the problem required manual intervention, either at the server (Worldtalk) or the client LAN system (SyncWare).

Mail*Hub's approach was to identify the user name and shunt it off into a part of the database--purely a manual operation. CDS gave us other ways to solve this, all of which promised to involve a great deal of programming. Our test user, John Smallberries, ended up in the X.500 directory twice, once as a normal person and once as a user in the special "hidden" organizational unit (OU).

Our third case looks the same to all packages: two users, one of which goes into the "hidden" part and doesn't get propagated. With SyncWare, we had to go to the client system and reconfigure the agent software--a straightforward but cumbersome process.

Worldtalk was easier, but its GUI interface only lets you do common tasks. For difficult or unusual operations, you have to drop down to a command line interface, even more cryptic than the usual Unix command set.

In this case, John O'Conner (who also appeared as "J. O'Conner"), led us on a merry trip down Mail*Hub's custom programming options, investigating what it would take to modify the import mechanism to handle everyone in a post office who only has an initial as their first name. We began to understand why Mail*Hub is only sold with a considerable chunk of consulting expertise.

Our last case was handled in almost the same way as before. We were hoping for some special handling--no cigar. To truly shield the non-propagating user from any query, we also had to modify the X.500 access control lists (ACLs) so that a database lookup would not return the hidden users.


February 13, 1996