Data Modeler Application Components

  • Data Modeler:  Used to create and manage metadata models.  This is the primary interface to all of the Data Modeler services.

  • DataBridge:  Manages data transfers for models which stage Notes data to relational databases.

  • Metadata Library:  Aggregates all metadata definitions across models and Data Modeler instances.

  • Sample Data:  A live Notes application to use for Data Modeler tutorials and test models.

  • Data Modeler Help:  Online knowledge base including installation and use of the product.

Access the Notes Application from the BI Application

  1. If using External Staging, DataBridge automates and schedules the transfer of data from the target application to a relational database.
  2. Use one of the studios in Cognos to build a report using the published metadata package.
  3. If using Direct Access, the BI server queries directly against the live target application.
  4. If using External Staging, the Cognos Server queries the image of the application data stored in the relational database.
  5. Reports are stored on the BI server.

Create a Metadata Model

  1. Run the Data Modeler from the Notes client on the Modeling Workstation.
  2. Select a target Notes Application on any Domino server.
  3. Run the Data Modeler Wizard to create a metadata model for the Notes application.
  4. A Model Definition File is written to the file system on the Modeling Workstation.
  5. Data Access Views are written to the target application.
  6. If External Staging is specified, transfer documents are written to the DataBridge application.

Install and Configure

Domino Server:

  • Install the Data Modeler applications on the Domino Server.
  • Sign the Data Modeler databases with a user ID that has full admin rights.
  • Install the DomSQL JDBC service

Database Server:

  • Create a Database for staged data.

Modeling Workstation:

  • Install the Notes Client with a Notes ID that has Designer access to the Notes application.
  • Install the Cognos Framework Manager and Configure it to access the Cognos server
  • Install the DomSQL JDBC driver into Framework Manager
  • Install and configure the database client

Cognos BI Server:

  • Install and configure the database client
  • Install the DomSQL JDBC driver
  • Create a Cognos Data Source for the staged data database.
     

Data Access / Modeling Strategies

There are two primary strategies for accessing Notes data using the Data Modeler for Notes.  Direct Access is where data is accessed directly in the NSF file.  Data Staging is where Notes data is written, or staged, to a relational database.  Either strategy may be more appropriate for the specific reporting requirement at hand.

As an example, a call center may want to report on both open and closed call tickets.  There are relatively few open call tickets, but they are updated constantly.  Real time displays such as management dashboards require real-time access to the live data.  This calls for a Direct Access strategy to the live Notes data.  

There are many, many more closed call tickets, and they are rarely, if ever, updated.  It may make more sense to stage closed call ticket data to a relational database, where more complex analytics such as OLAP can be efficiently done.  

The call center selects different access strategies to report on the different types of call tickets.  The Data Modeler provides all of the features the call center needs to optimize their data access to provide a seamless Business User experience.

Direct Access:  Direct Access uses a driver to query the Notes data directly where it resides in the Notes database and make the data structure look like a relational data structure.  This approach is fast, efficient, and allows reporting in real time on documents that are frequently edited.  

The Data Modeler supports the legacy Lotus NotesSQL ODBC driver and also includes the new DomSQL JDBC driver for Domino.  The ODBC driver suffers from performance limitations and is suitable for relatively small data sets.  The DomSQL JDBC driver offers much faster performance as well as a much more capable feature set.  JDBC should now be the option of choice for virtually all direct access needs.  

Whether using ODBC or JDBC, query performance will begin to degrade when accessing larger sets of data.  This is due to the inherent limitations of the Notes NSF database design.  If you are accessing data sets in excess of 100,000 documents or performing complex analytics on the data, You should consider staging the data to a relational database.

External Staging:  External Staging uses the Data Modeler DataBridge to stage an image of the data from the Notes database to a relational database.  This snapshot of the Notes data is then accessed by BI applications.  This method is ideal for large sets of data where the documents are not frequently edited.  The data can be staged manually or on a schedule, but the staged data represents the content of the Notes database as of the last transfer event.  For many business reporting needs, a snapshot as of the end of the last business day works fine.

External Staging is also used to integrate Notes data with a Data Warehouse.

Hybrid Solution:  Both of the above options can be integrated into a hybrid solution that combines the strengths of both strategies.  Direct Access models can be created for dynamic sets of data that require real time reporting on documents that are often edited.  Larger, infrequently edited sets of data can be externally staged to a relational data store, with a separate model created to report directly on that data store.

In the BI application, the two models can be joined using common key values for seamless and optimized reporting on the entire set of data.