IBM® Lotus Domino attachment and object service (DAOS)

The IBM® Lotus® Domino® attachment and object service (DAOS) in release 8.5 reduces the total cost of ownership and helps customers with green computing practices by storing all file attachments in a separate repository on the server and retrieving them by reference. Read more to plan, set up, configure, and manage Lotus Notes® large objects.



With the release of version 8.5, IBM Lotus Domino server employs the Domino attachment and object service to save significant space at the file level by sharing data identified as identical between databases (applications) on the same server. Document attachments are the first components to use the DAOS feature in Lotus Domino.

In databases that use DAOS, Lotus Domino no longer saves a separate and complete copy of every document attachment. Instead, the server saves a reference to each attached file in an internal repository, and it refers to the same file from multiple documents in one or more databases on the same server. When an attached file is large and a message containing it is broadcast to thousands of users, creating a separate copy of the message for each recipient could require several gigabytes of disk space. Multiple copies of the same attachment often also proliferated in mail threads with multiple replies. With DAOS enabled, disk space usage is substantially reduced.

Use of an attachment object store is optional, and it involves considerable planning before you can implement it in Lotus Domino.

You can mark databases on a Lotus Domino server for participation in attachment consolidation by enabling consolidation on the DAOS tab in the Server document, and by ensuring that every database that you want to include in consolidation has the "Use Domino Attachment and Object Service" advanced database property selected. DAOS also requires transaction logging to be enabled. DAOS stores a single copy of each attachment in a central mapped repository. After you enable attachment consolidation on a server, all databases on the server that are included in consolidation use the repository to store attachments.

When attachment consolidation is enabled and a user saves an attachment, the body stored in the document contains a reference, sometimes called a ticket, to the attachment, which identifies the attachment in the repository. Consolidation occurs immediately; you do not have to wait for a task to run before disk space savings occur in the size of documents with attachments.

DAOS

Developer Works Article on DAOS

There's a newly published article on DeveloperWorks that outlines how to setup DAOS in your environment. In the meantime, take a look here for more information.


DAOS How it Works and Security

In this section, we'll talk about how DAOS works and how it is secured.


So, first of all, DAOS will work on ANY database that resides on a DAOS-enabled server. There is a property selection box in the database properties, and if enabled, the database will use DAOS for all it's attachments. Basically, what happens is this:

When a document is saved (or emailed, or whatever), Domino sees it as essentially
ddddddXXXXXXXXXddddddddddddXXXXXXXXXXXXXXXdddddddddddddXXXX
where "d" represents the body and "X" represents one or more attachments.

DAOS "rewrites" that so that Domino now sees the document as
ddddddTddddddddddddTdddddddddddddT
where "T" is the "small ticket" information for DAOS

Then, DAOS puts the attachments in the file system and also puts a counter/reference to those attachments in a DAOS Catalog nsf file (more on that feature in a moment). You would have an NLO file for each attachment in the document (as long as the attachments are DIFFERENT).

There you have it! Now, you have a bunch of .NLO files on the file system of your Domino server. Then, when a user opens the document and double-clicks on the attachment icon, Domino knows to go to the DAOS store and retrieves the attachment.

But WAIT, you say...How do I secure it? Can't anyone just get into those .NLO files and manipulate them?

Well, yes and no. First of all, they are on the file system of your Domino server, and a user can't access those files in any way other than through the file structure. So let's take a moment and talk about how secure your Domino server is. In theory, if people have access to the file structure of your Domino server, you have more to worry about than them looking at those .NLO files and reading attachments! They have access to EVERYTHING! The keys to the kingdom, so to speak! They can access id files, .ini files not to mention EVERY single database on the server. So..I'm going to assume that your Domino server is locked down so that Joe user can't just map a drive to it and get at the files.

Secondly, in the next beta drop of 8.5, we will be providing an encryption mechanism for the DAOS store. Therefore, all the files will be encrypted. So, if Joe user does happen to have access, well, now they can be encrypted!

Now, back to how counts work and that comment above about attachments only being stored if they are different..

There is a database, the DAOS Catalog, that keeps track of all the counts for an attachment and where the "tickets" for the attachment are referenced. It knows every .NLO created, how many references for each of them and maintains a list of every .NSF file using the attachments. And, being a Notes database, if it becomes corrupted, DAOS will detect that corruption and attempt to remain operable. But, if for some reason the corruption is such that DAOS can't continue to function, there will be some commands an administrator can run that will resync everything.

Suffice to say, the developers will ensure you can get at your attachments! There will be many tools you can leverage that will allow you to restore NLOs, fixup the stores and keep the store up to date. Having said that however, you can't really manipulate the DAOS counts on your own. Administratively, it's a no-no.

We also had some great questions about the fact that if you got spammed or did a copy/paste of an attachment, wouldn't there be a million files out in the DAOS store? Now, here's where it gets really cool.

When you do a copy/paste of an attachment or if an attachment is the same across multiple messages, the DAOS code recognizes that! DAOS will then only store one version of the attachment, and create a ton of reference counts for each document!

So, while we can't keep you from getting spammed or copying attachments a bunch of times, we can make it easier by saving you a lot of disk space when that occurs! Too cool!

WHEW! That's a LOT of information!!!