The metadata inference API allows administrators to automate the process of collecting and entering metadata about variables in their data. The inference engine allows administrators to browse, configure, and run agents that will evaluate the data and document what it finds.
Getting Available Agents
Different agents can apply to different data sources / views. Agents for a specific view can be discovered as follows.
http://{host}/rds/api/admin/catalog/{collection}/{view}/agents
[
{
"licensed": "true", // indicates whether or not the RDS license currently installed supports this agent
"label": "Frequency Agent",
"id": "us.mtna.core:FrequencyAgent"
},
{
"licensed": "true",
"label": "Frequency Agent (RDS Query Based)",
"id": "us.mtna.rds:FrequencyAgent"
},
{
"licensed": "false",
"label": "Missing Value Agent",
"id": "us.mtna.core:MissingValueAnalysisAgent"
},
{
"licensed": "true",
"label": "Summary Statistics Agent",
"id": "us.mtna.core:SummaryStatisticsAgent"
},
{
"licensed": "false",
"label": "Unique Values Agent",
"id": "us.mtna.core:UniqueValueAnalysisAgent"
}
]
Getting Properties for a Specific Agent
Once the user decides which agent(s) they want to use they can retrieve the properties for that specific agent using the agent ID as follows:
http://{host}/rds/api/admin/catalog/{collection}/{view}/agents/us.mtna.core:FrequencyAgent
[
{
"description": "The name of the agent.",
"label": "Name",
"name": "name",
"type": "TEXT",
"value": "Frequency Agent",
"system": false,
"repeatable": false,
"required": false,
"readOnly": true
},
{
"description": "The ID of the agent.",
"label": "Agent ID",
"name": "id",
"type": "TEXT",
"value": "us.mtna.core:FrequencyAgent",
"system": false,
"repeatable": false,
"required": true,
"readOnly": true
},
{
"description": "The variables for the agent to analyze.",
"label": "Variables",
"name": "variables",
"type": "TEXT",
"value": "", // fill out if desired
"system": false,
"repeatable": true,
"required": false,
"readOnly": false
},
{
"description": "The variables to use as weights.",
"label": "Weights",
"name": "weights",
"type": "TEXT",
"value": "", // fill out if desired
"system": false,
"repeatable": true,
"required": false,
"readOnly": false
}
]
Running a Job
Once one or more agents have been chosen their properties can be retrieved and filled out as needed. When they are filled out they can be added to an array and posted to the server. Once the properties are posted to the server a job will be created and the job ID will be returned. This can be used for checking the status of the job.
http://{host}/rds/api/admin/catalog/{collection}/{view}/run/agents
[
[
{
"description": "The name of the agent.",
"label": "Name",
"name": "name",
"type": "TEXT",
"value": "Frequency Agent",
"system": false,
"repeatable": false,
"required": false,
"readOnly": true
},
{
"description": "The ID of the agent.",
"label": "Agent ID",
"name": "id",
"type": "TEXT",
"value": "us.mtna.core:FrequencyAgent",
"system": false,
"repeatable": false,
"required": true,
"readOnly": true
},
{
"description": "The variables for the agent to analyze.",
"label": "Variables",
"name": "variables",
"type": "TEXT",
"value": "", // fill out if desired
"system": false,
"repeatable": true,
"required": false,
"readOnly": false
},
{
"description": "The variables to use as weights.",
"label": "Weights",
"name": "weights",
"type": "TEXT",
"value": "", // fill out if desired
"system": false,
"repeatable": true,
"required": false,
"readOnly": false
}
]
]
Viewing a Job's Status
Once a job is submitted to RDS, processes will be run on the server to run the agents and document their findings as "facts" available on the variables of the view. Each job will have information about the status of each of the agents that it has run. To view the job's status use its job ID as follows:
http://{host}/rds/api/admin/jobs/7fcd879d-c0f3-4a50-9f0c-6bfe67fcb237
{
"status": "COMPLETED",
"completed": 100,
"processDetails": {
"Frequency Agent": {
"id": "32e7173e-2f04-4c67-9d48-e6bfb50b1a28",
"status": "COMPLETED",
"completed": 100,
"completionDetails": {
"collection": "anes",
"dataSource": "anes1948",
"factsFound": 67
}
}
}
}
Viewing Facts
Once a job has been completed, facts should be applied to the applicable variables.
http://{host}/rds/api/admin/catalog/{collection}/{view}/variables?facts
{
"variables": {
"resources": [
{
"id": "VAR1",
"name": "VAR1",
"index": 1,
"storageType": "NUMERIC",
"displayType": "NUMERIC",
"width": 10,
"facts": [
{
"source": "us.mtna.core:FrequencyAgent",
"resourceUris": [
"Variable:VAR1"
],
"values": [
{
"frequencies": {
"1": 182,
"2": 354,
"3": 126
},
"dataCoverage": "EXHAUSTIVE"
}
],
"valueClasses": [
"us.mtna.rds.services.model.metadata.Frequencies"
],
"timestamp": "2016-10-03T12:22:19.083-08:00[America/Anchorage]"
}
]
},
...
]
}
}