datamanager-1.0.0.zip
(Windows) or
datamanager-1.0.0.tar.gz
(Linux or Unix).
The Java application that uses this library (referred to throughout this document as the Calling
Application) should include the datamanager.jar
file in its Java classpath.
Use Case | Capability | DataManager Method Name |
---|---|---|
1 | Parse a metadata document to obtain information about the entities and attributes in the data package |
parseMetadata()
|
2 | Download data from the remote source to a local data store |
downloadData()
|
3 | Load data into a relational database table; supported relational database management systems are HSQL, Oracle, and PostgreSQL |
loadDataToDB()
|
4 | Query the data from the relational database |
selectData()
|
The Calling Application is required to implement a class containing methods to get a database connection from a connection pool, return a database connection to the connection pool, and get the database adapter name corresponding to the RDBMS being used by the application.
An example of an implementing class can be found in src/org/ecoinformatics/datamanager/sample/SampleCallingApp.java described in the Sample Calling Application section below.
The Calling Application is required to implement a class containing methods to interact with the Data Manager Library to download data to the local data store. Although the Data Manager Library is responsible for downloading data from the remote site to the local data store, it makes no assumptions about where to store data in the local data store or how the data store should be managed. The Data Manager Library relies on the Calling Application to provide it with a set of methods that can be called to start serialization of the data, finish serialization of the data, and determine whether the data already exists in the local store.
An example of an implementing class can be found in src/org/ecoinformatics/datamanager/sample/SampleDataStorage.java described in the Sample Calling Application section below.
The Calling Application is required to implement a class containing methods to provide a Metacat Ecogrid end point, provide a SRB Ecogrid end point, and provide a SRB machine name.
An example of an implementing class can be found in src/org/ecoinformatics/datamanager/sample/EcogridEndPoint.java described in the Sample Calling Application section below.
org.ecoinformatics.datamanager.sample
. It is configured to
load database connectivity properties and other properties at run-time
from file datamanager.properties (as accessible in your classpath). The user may edit this
file and modify these properties in accordance with local database settings.
The sample Calling Application consists of three classes, described in the table below:
Class | Purpose | Implements Interface |
---|---|---|
SampleCallingApp
|
Main program for the sample Calling Application. Executes a number of small tests to demonstrate the use cases supported by the Data Manager Library API. |
DatabaseConnectionPoolInterface
|
SampleDataStorage
|
Demonstrates implementation of DataStorageInterface |
DataStorageInterface
|
EcogridEndPoint
|
Demonstrates implementation of EcogridEndPointInterface |
EcogridEndPointInterface
|
To run the sample Calling Application, change directory to the top-level of the Data Manager Library distribution (the directory that contains the datamanager.jar file). For example:
cd C:\datamanager-1.0.0
Next, run the following command:
java -cp "datamanager.jar" org.ecoinformatics.datamanager.sample.SampleCallingApp
If it executes successfully, the output of the sample Calling Application will look similar to the following:
Finished testParseMetadata(), success = true
Constructing DownloadHandler for URL: http://gce-lter.marsci.uga.edu/lter/asp/db/send_file.asp?name=metacat-user&email=none&affiliation=LNO¬ify=0&accession=INS-GCEM-0011&filename=INS-GCEM-0011_1_3.TXT
the identifier is ============ tao2075037663
the identifier is ============ tao2075037663
Finished testDownloadData(), success = true
[Ljava.lang.String;@16cd7d5
[Ljava.lang.String;@cdedfd
Attribute Name: Site
DB Field Name : Site
dbDataType : TEXT
Attribute Name: Year
DB Field Name : Year
dbDataType : TEXT
Attribute Name: Month
DB Field Name : Month
dbDataType : TEXT
Attribute Name: Day
DB Field Name : Day
dbDataType : TEXT
Attribute Name: Transect
DB Field Name : Transect
dbDataType : TEXT
Attribute Name: Species_Code
DB Field Name : Species_Code
dbDataType : TEXT
Attribute Name: Count
DB Field Name : Count
dbDataType : INTEGER
Constructing DownloadHandler for URL: http://gce-lter.marsci.uga.edu/lter/asp/db/send_file.asp?name=metacat-user&email=none&affiliation=LNO¬ify=0&accession=INS-GCEM-0011&filename=INS-GCEM-0011_1_3.TXT
Finished testLoadDataToDB(), success = true
Query SQL = 'SELECT INS_GCEM_0011_1_3_TXT.Count FROM INS_GCEM_0011_1_3_TXT where INS_GCEM_0011_1_3_TXT.Count > 1;'
Printing all records with 'count' value greater than 1
resultSet[1], count = 2
resultSet[2], count = 3
resultSet[3], count = 2
resultSet[4], count = 2
resultSet[5], count = 2
resultSet[6], count = 4
resultSet[7], count = 2
resultSet[8], count = 3
resultSet[9], count = 5
resultSet[10], count = 8
resultSet[11], count = 5
resultSet[12], count = 8
resultSet[13], count = 5
resultSet[14], count = 9
resultSet[15], count = 7
Finished testSelectData(), success = true
tableName: INS_GCEM_0011_1_3_TXT
fieldNames[0]: site
fieldNames[1]: year
fieldNames[2]: month
fieldNames[3]: day
fieldNames[4]: transect
fieldNames[5]: species_code
fieldNames[6]: count
Finished testEnumerationMethods(), success = true
Finished all tests, success = true
Finished dropping tables.
Use Case | Capability | DataManager Method Name |
---|---|---|
5 | Set an upper limit on the size of the database. The Data Manager Library will monitor the size of the database, and if the upper limit (as set by the Calling Application) is exceeded, old data tables will be dropped as needed to free up space in the database. |
setDatabaseSize()
|
6 | Set a life-span priority on individual data tables. This relates to the previous capability (Use Case 5), in that the Calling Application may single out individual data tables as high priority to indicate that they should not be dropped when the database exceeds the specified size limit. |
setTableExpirationPolicy()
|