<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Event Logging and Reporting — v2.1.0-beta</title>
<link rel="stylesheet" href="../_static/dataone.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '../',
VERSION: '2.1.0-beta',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true,
SOURCELINK_SUFFIX: '.txt'
};
</script>
<script type="text/javascript" src="../_static/mathjax_pre.js"></script>
<script type="text/javascript" src="../_static/jquery.js"></script>
<script type="text/javascript" src="../_static/underscore.js"></script>
<script type="text/javascript" src="../_static/doctools.js"></script>
<script type="text/javascript" src="//cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-MML-AM_CHTML"></script>
<script type="text/javascript" src="../_static/sidebar.js"></script>
<link rel="author" title="About these documents" href="../about.html" />
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
<link rel="next" title="Logging Schema" href="LoggingSchema.html" />
<link rel="prev" title="Serialization of Types for Transfer Over HTTP" href="Serialization.html" />
<link media="only screen and (max-device-width: 480px)" href="../_static/small_dataone.css" type= "text/css" rel="stylesheet" />
</head>
<body role="document">
<div class="version_notice">
<p>
<span class='bold'>Warning:</span> These documents are under active
development and subject to change (version 2.1.0-beta).<br />
The latest release documents are at:
<a href="https://purl.dataone.org/architecture">https://purl.dataone.org/architecture</a>
</p>
</div>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="right" >
<a href="../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="right" >
<a href="LoggingSchema.html" title="Logging Schema"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="Serialization.html" title="Serialization of Types for Transfer Over HTTP"
accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../index.html"></a> »</li>
<li class="nav-item nav-item-1"><a href="index.html" accesskey="U"><no title></a> »</li>
</ul>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body">
<div class="section" id="event-logging-and-reporting">
<h1>Event Logging and Reporting<a class="headerlink" href="#event-logging-and-reporting" title="Permalink to this headline">¶</a></h1>
<p>The DataONE system should log various interactions and operations in the system
to provide operational status information about the entire system, to report on
specific node operations, and to inform DataONE participants (users,
contributors, administrators) about their specific domain of interest in the
system. For example, a contributor might like to monitor use of their data and
where it is being replicated to. The methods <a class="reference internal" href="../apis/MN_APIs.html#MNCore.getLogRecords" title="MNCore.getLogRecords"><code class="xref py py-func docutils literal"><span class="pre">MNCore.getLogRecords()</span></code></a> and
<a class="reference internal" href="../apis/CN_APIs.html#CNCore.getLogRecords" title="CNCore.getLogRecords"><code class="xref py py-func docutils literal"><span class="pre">CNCore.getLogRecords()</span></code></a> provide outward facing services for retrieving
log information from member and coordinating nodes respectively.</p>
<p>Logging is described in use cases 16, 17, 18, 20, and potentially 19.</p>
<div class="toctree-wrapper compound">
<ul>
<li class="toctree-l1"><a class="reference internal" href="LoggingSchema.html">Logging Schema</a></li>
</ul>
</div>
<div class="section" id="use-cases-to-be-supported">
<span id="logging-use-case-synopsis"></span><h2>Use Cases to be Supported<a class="headerlink" href="#use-cases-to-be-supported" title="Permalink to this headline">¶</a></h2>
<ul class="simple">
<li>UC 16 specifies “all CRUD operations on metadata and data are logged at each
node”</li>
<li>UC 17 indicates that all CRUD operation logs should be aggregated at the CNs</li>
<li>UC 18 indicates that a MN can retrieve aggregated logs about content that
originated from that MN.</li>
<li>UC 20 indicates that a data owner (original contributor, delegated owner)
can retrieve aggregated logs about objects they own.</li>
<li>UC 19 indicates that anyone can retrieve general use information for any
object in DataONE.</li>
</ul>
</div>
<div class="section" id="performance-metrics-to-be-reported">
<h2>Performance Metrics to be Reported<a class="headerlink" href="#performance-metrics-to-be-reported" title="Permalink to this headline">¶</a></h2>
<p>The performance metrics survey results from the Leadership Team specify (at
least) the following metrics should be captured. Items that may be captured
from the CI portion of the project are indicated by !!!.</p>
<p><strong>Size and Diversity of DataONE Data, Metadata, and Investigator Toolkit
Holdings</strong></p>
<ol class="arabic simple">
<li>!!! Data volume – total size of data holdings in DataONE Member Nodes and
Coordinating Nodes</li>
</ol>
<blockquote>
<div><dl class="docutils">
<dt>Recorded in:</dt>
<dd><ul class="first last simple">
<li>Total (including replicas) data volume + unique object data volume</li>
<li>Sysmeta</li>
</ul>
</dd>
</dl>
</div></blockquote>
<ol class="arabic simple" start="2">
<li>!!! Number of metadata records – quantity of metadata records held at a
Coordinating Node (note: the concept of a record may vary across metadata
standards)<ul>
<li>science metadata only</li>
</ul>
</li>
<li>!!! Number of data sets held by Member Nodes – (note: may be less
meaningful as an absolute value because of the immense variability in data set
granularity, but probably still useful in tracking the shape of the data
accumulation curve)</li>
</ol>
<blockquote>
<div><ul class="simple">
<li>system metadata</li>
</ul>
</div></blockquote>
<ol class="arabic simple" start="4">
<li>!!! Number and types of software tools included in the Investigator Toolkit</li>
</ol>
<blockquote>
<div><ul class="simple">
<li>!!</li>
<li>Mechanism to register external uses / implementations</li>
<li>HTTP user-agent</li>
</ul>
</div></blockquote>
<ol class="arabic simple" start="5">
<li>!!! Number of different metadata schemas supported – (note: more metadata
schemas is not necessarily better)</li>
</ol>
<blockquote>
<div><ul class="simple">
<li>object format from sysmeta</li>
</ul>
</div></blockquote>
<p><strong>DataONE System Capacity</strong></p>
<ol class="arabic simple" start="6">
<li>!!! Number of Member Nodes</li>
</ol>
<blockquote>
<div><ul class="simple">
<li>registry</li>
</ul>
</div></blockquote>
<ol class="arabic simple" start="7">
<li>!!! Total storage capacity at Member Nodes</li>
</ol>
<blockquote>
<div><ul class="simple">
<li>per member node</li>
<li>total</li>
<li>part of MN registration and capabilities</li>
</ul>
</div></blockquote>
<ol class="arabic simple" start="8">
<li>!!! Geographic coverage of Member Nodes – continents, regions, and countries
covered<ul>
<li>could potentially be part of capabilities (record physical location)</li>
</ul>
</li>
<li>!!! Number of Coordinating Nodes<ul>
<li>constant = 3</li>
<li>general expansion over five years to all countries.</li>
</ul>
</li>
<li>!!! Total storage capacity at Coordinating Nodes</li>
</ol>
<blockquote>
<div><ul class="simple">
<li>disk space</li>
</ul>
</div></blockquote>
<ol class="arabic simple" start="11">
<li>!!! Geographic coverage of Coordinating Nodes – continents, regions, and
countries covered</li>
</ol>
<blockquote>
<div><ul class="simple">
<li>capabilities / metadata</li>
</ul>
</div></blockquote>
<p><strong>DataONE Usage Statistics</strong></p>
<ol class="arabic simple" start="12">
<li>!!! (CN LOG) Number of web hits on DataONE portal</li>
</ol>
<blockquote>
<div><ul class="simple">
<li>standard web hits from logs</li>
<li>aggregated across CNs</li>
</ul>
</div></blockquote>
<ol class="arabic simple" start="13">
<li>!!! (CN LOG) Number of DataONE users – (note: recording of individual IP
addresses may be most readily implemented; requiring users to login to Member
Nodes is not presently required)<ul>
<li>Any users of MNs = D1 user</li>
<li>Eventually can use authn subsystem</li>
</ul>
</li>
<li>!!! (supporting web site logs) Number of downloads of tools from the
Investigator Toolkit –> “Number of times tools are downloaded”<ul>
<li>Log analysis</li>
<li>Text needs to be clarified. ITK isn’t a place</li>
<li>supporting information (videos, documents, ...)</li>
</ul>
</li>
<li>!!! (CN LOG) Number of metadata catalog searches completed – (note: over
time it may also be desirable to assess precision and recall of incoming
searches)<ul>
<li>part of the Mercury log (in mysql)</li>
</ul>
</li>
<li>!!! (MN, CN LOG) Number of DataONE datasets downloaded (daily, weekly,
monthly, annually) – (note: this may be straightforward if included in
specifications for Member Node data, impossible otherwise)<ul>
<li>Member node access logs - need to be aggregated</li>
<li>What was pulled through D1.get() vs the native mechanisms of the MN</li>
<li></li>
</ul>
</li>
<li>!!! (MN, CN LOG) Most frequently downloaded datasets</li>
</ol>
<blockquote>
<div><ul class="simple">
<li>Same as 16</li>
</ul>
</div></blockquote>
<p><strong>Reliability and System Performance</strong></p>
<ol class="arabic simple" start="18">
<li>!!! (CN heartbeat logs) Uptime (availability) of Coordinating Nodes</li>
</ol>
<blockquote>
<div><ul class="simple">
<li>Need a monitoring service in addition to the CN service</li>
<li>also need to consider geographic accessibility (users)</li>
</ul>
</div></blockquote>
<ol class="arabic simple" start="19">
<li>!!! (MN heartbeat logs) Uptime (availability) of Member Nodes – (note:
tracked if heartbeat is fully implemented)</li>
</ol>
<blockquote>
<div><ul class="simple">
<li>Same as 18</li>
</ul>
</div></blockquote>
<ol class="arabic simple" start="20">
<li>!!! Server response time</li>
</ol>
<blockquote>
<div><ul class="simple">
<li>REST service performance</li>
<li>Define a bunch of test queries that can be executed in parallel for load testing.</li>
</ul>
</div></blockquote>
<ol class="arabic simple" start="21">
<li>!!! Response time of user interface</li>
</ol>
<blockquote>
<div><ul class="simple">
<li>Time for “page load” vs. number of concurrent users</li>
<li>Time for specific operations (test queries, test renderings, ...)</li>
</ul>
</div></blockquote>
<p><strong>Community Engagement</strong></p>
<ol class="arabic simple" start="22">
<li>Baseline assessment of scientists completed</li>
<li>Number of repeat assessments of scientists completed</li>
<li>Baseline assessment of other stakeholders completed</li>
<li>Number of repeat assessments of other stakeholders completed</li>
<li>Number of DataONE Partnership Agreements established</li>
</ol>
<p><strong>Education and Outreach</strong></p>
<ol class="arabic simple" start="27">
<li>Number of education modules developed and/or accessible through DataONE</li>
<li>Number of times education modules are downloaded</li>
<li>Number of best practices guides developed and/or accessible through DataONE</li>
<li>Number of times best practices guides are downloaded</li>
<li>Number of training sessions or workshops offered (e.g., at meetings)</li>
<li>Number of workshop participants</li>
<li>Number of people in DataONE International Users Group</li>
</ol>
<p><strong>Sustainability</strong></p>
<ol class="arabic simple" start="34">
<li>Amount of revenue (including in-kind support) generated annually to
support DataONE</li>
<li>Diversity of revenue streams – e.g., government, private foundations,
commercial for-profit sector, etc.</li>
<li>Number of projects and partners collaborating with DataONE or leveraging
DataONE infrastructure</li>
</ol>
</div>
<div class="section" id="union-of-use-cases-and-metrics">
<h2>Union of Use Cases and Metrics<a class="headerlink" href="#union-of-use-cases-and-metrics" title="Permalink to this headline">¶</a></h2>
<p>The following bullets represent the union of logging information indicated in
the use cases and the metrics that can be reported from the logs. The
information logged and suitable summarization and extraction procedures need
to be identified to ensure the following items can be addressed:</p>
<ul class="simple">
<li>all CRUD operations on metadata and data are logged at each node</li>
<li>all CRUD operation logs should be aggregated at the CNs</li>
<li>an MN can retrieve aggregated logs about content that originated from that
MN.</li>
<li>A data owner (original contributor, delegated owner) can retrieve aggregated
logs about objects they own.</li>
<li>Anyone can retrieve general use information for any object in DataONE.</li>
<li>(metric) Number of web hits on DataONE portal</li>
<li>(metric) Number of DataONE users</li>
<li>(metric) Number of downloads of tools from the Investigator Toolkit (From
the download site logs)</li>
<li>(metric) Number of metadata catalog searches completed</li>
<li>(metric) Number of DataONE datasets downloaded (daily, weekly, monthly,
annually)</li>
<li>(metric) Most frequently downloaded datasets</li>
</ul>
</div>
</div>
</div>
</div>
</div>
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
<div class="sphinxsidebarwrapper">
<p class="logo"><a href="http://dataone.org">
<img class="logo" src="../_static/dataone_logo.png" alt="Logo"/>
</a></p>
<h3><a href="../index.html">Table Of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">Event Logging and Reporting</a><ul>
<li><a class="reference internal" href="#use-cases-to-be-supported">Use Cases to be Supported</a></li>
<li><a class="reference internal" href="#performance-metrics-to-be-reported">Performance Metrics to be Reported</a></li>
<li><a class="reference internal" href="#union-of-use-cases-and-metrics">Union of Use Cases and Metrics</a></li>
</ul>
</li>
</ul>
<h3>Related Topics</h3>
<ul>
<li><a href="../index.html">Documentation Overview</a><ul>
<li><a href="index.html"><no title></a><ul>
<li>Previous: <a href="Serialization.html" title="previous chapter">Serialization of Types for Transfer Over HTTP</a></li>
<li>Next: <a href="LoggingSchema.html" title="next chapter">Logging Schema</a></li>
</ul></li>
</ul></li>
</ul>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
<form class="search" action="../search.html" method="get">
<div><input type="text" name="q" /></div>
<div><input type="submit" value="Go" /></div>
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="footer">
<div id="copyright">
© Copyright <a href="http://www.dataone.org">2009-2017, DataONE</a>.
[ <a href="../_sources/design/logging.txt"
rel="nofollow">Page Source</a> |
<a href='https://redmine.dataone.org/projects/d1/repository/changes/documents/Projects/cicore/architecture/api-documentation/source/design/logging.txt'
rel="nofollow">Revision History</a> ]
</div>
<div id="acknowledgement">
<p>This material is based upon work supported by the National Science Foundation
under Grant Numbers <a href="http://www.nsf.gov/awardsearch/showAward?AWD_ID=0830944">083094</a> and <a href="http://www.nsf.gov/awardsearch/showAward?AWD_ID=1430508">1430508</a>.</p>
<p>Any opinions, findings, and conclusions or recommendations expressed in this
material are those of the author(s) and do not necessarily reflect the views
of the National Science Foundation.</p>
</div>
</div>
<!--
<hr />
<div id="HCB_comment_box"><a href="http://www.htmlcommentbox.com">HTML Comment Box</a> is loading comments...</div>
<link rel="stylesheet" type="text/css" href="_static/skin.css" />
<script type="text/javascript" language="javascript" id="hcb">
/*<! -*/
(function()
{s=document.createElement("script");
s.setAttribute("type","text/javascript");
s.setAttribute("src", "http://www.htmlcommentbox.com/jread?page="+escape((typeof hcb_user !== "undefined" && hcb_user.PAGE)||(""+window.location)).replace("+","%2B")+"&mod=%241%24wq1rdBcg%24Gg8J5iYSHJWwAJtlYu/yU."+"&opts=21407&num=10");
if (typeof s!="undefined") document.getElementsByTagName("head")[0].appendChild(s);})();
/* ->*/
</script>
-->
</body>
</html>