The csv connector ships with every HIRO instance and enables you to create Automation issues and upload environment and monitoring data.\nIt makes use of HIROs Rest API using CSV files as input. We’ve put together sample csv files for each scenario to help you get started.
\nA token is mandatory to use any of the capabilities the csv connector offers. To retrieve a token, you must run a Post request against https://core.engine.datagroup.de/api/auth/6/app.
\n{\n \"client_secret\": \"ffc55c735c79d22441e941b09fbe9afbda839dc12012b7bd37f9d88b54fd13add74fdef8abf8194e9729a281bfbe68935950fef7a9dd690827082fc611f68492\",\n \"client_id\": \"cju16o7cf0000mz77pbwbhl3q_ckkwosk448atc01543wvzxvb4\",\n \"username\":\"\",\n \"password\": \"\"\n}Username and password is created for every HIRO 7 instance that is provisioned for you. The login information is generated and can be found on id.almato.ai. The organisation admin will be able to change the pregenerated password for the user that is called haas*-csv-connector.\nThe scope id, an id that identifies which HIRO 7 instance the csv content can be found doing a GET request with the token from the username above against https://core.engine.datagroup.de/api/auth/6/me/account?profile=true\nThe scope can be found in \"ogit/Auth/Account/defaultScope\"
\nThe CSV Connector works with HTTP requests. All capabilities share the same base url (https://api.hiro.almato.ai/function/csv-connector) followed by the action after the slash, such as /vertices /edges /timeseries or /attachments. You can either run those request from command line or any tool of your choice (such as postman for instance)
\nPlease make sure to add the following header information to your requests
\nx-forwarded-authorization: Bearer ${your_token}\ncontent-type: text/csv\nAdditional header parameters control details of csv handling within the connector:
\n| Header | \nDescription | \n
|---|---|
| \nSet the csv dialect for the csv parser ( | \n
| \nBoolean value. When set to | \n
| \nTake the string given here as null keyword to set attributes for HIRO explicitly to None. | \n
With the token in hand you can run a POST request against.
\nusing the following sample CSV file. Please make sure to add a value for scope. If you want to create more than one object, you can add more lines to the csv file.\nKeep in mind POST does not allow you to update existing objects. Please use a PUT request if you want to update.
\n\"ogit/_xid\",\"ogit/_type\",\"ogit/MARS/Machine/class\",\"ogit/name\",\"ogit/MARS/Software/class\",\"ogit/MARS/Software/subClass\",\"ogit/_scope\"\n\"machine1\",\"ogit/MARS/Machine\",\"Linux\",\"machine a\",\"\",\"\",\"cjuwixjvq0xfc1q90xqn8jsvf_ckcd798hd7bji0r882l764ebl\"\n\"software1\",\"ogit/MARS/Software\",\"\", \"Software A\",\"DBMS\",\"MariaDB\",\"cjuwixjvq0xfc1q90xqn8jsvf_ckcd798hd7bji0r882l764ebl\"With the token in hand you can run a POST request against\nhttps://api.hiro.almato.ai/function/csv-connector/vertices
\nusing the following sample CSV file. Please make sure to add a value for scope and origin node. The origin node defines where the issue is bound to first, where knowledge can be applied (if available). If you do not have an origin Node, please check part 1 where we show you how to create one.
\n\"ogit/subject\",\"ogit/Automation/originNode\",\"/ProcessIssue\",\"ogit/_type\",\"ogit/_scope\"\n\"This is a test Issue\",\"node_id\",\"process_me\",\"ogit/Automation/AutomationIssue\",\"scope_id\"With the token in hand you can run a PUT request against\nhttps://api.hiro.almato.ai/function/csv-connector/combined
\n\"ogit/_xid\",\"ogit/_type\",\"ogit/MARS/Machine/class\",\"ogit/name\",\"ogit/MARS/Software/classv,\"ogit/MARS/Software/subClass\",\"ogit/_scope\",\"issue:0:ogit/subject\",\"\"issue:0:/ProcessIssue\"\n\"machine1234\",\"ogit/MARS/Machine\",\"Linux\",\"machine a\",\"\",\"\",\"cjy5tvdfx0nkq1o7040e4mow0_ckd73u0ul04jo01100j5dppq7\",\"\",\"\"\n\"software1123\",\"ogit/MARS/Software\",\"\",\"Software A\",\"DBMS\",\"MariaDB\",\"cjy5tvdfx0nkq1o7040e4mow0_ckd73u0ul04jo01100j5dppq7\",\"test issue\",\"processme\"With the token in hand you can run a POST request against\nhttps://api.hiro.almato.ai/function/csv-connector/edges
\nusing the following sample CSV file.
\n\"from:ogit/_xid\",\"verb\",\"to:ogit/_xid\"\n\"software1\",\"ogit/dependsOn\",\"machine1\"With the token in hand you can run a PUT request against\nhttps://api.hiro.almato.ai/function/csv-connector/combined
\n\"ogit/_xid\",\"ogit/_type\",\"ogit/MARS/Machine/class\",\"ogit/name\",\"ogit/MARS/Software/class\",\"ogit/MARS/Software/subClass\",\"ogit/_scope\",\"edge:0:verb\",\"edge:0:direction\",\"edge:0:vertex_xid\"\n\"machine1\",\"ogit/MARS/Machine\",\"Linux\",\"machine a\",\"\",\"\",\"cjuwixjvq0xfc1q90xqn8jsvf_ckcd798hd7bji0r882l764ebl\",\"\",\"\",\"\"\n\"software1\",\"ogit/MARS/Software\",\"\",\"Software A\",\"DBMS\",\"MariaDB\",\"cjuwixjvq0xfc1q90xqn8jsvf_ckcd798hd7bji0r882l764ebl\",\"ogit/dependsOn\",\"out\",\"machine1\"First create a timeseries node\nWith the token in hand you can run a POST request against\nhttps://api.hiro.almato.ai/function/csv-connector/vertices
\n\"ogit/_xid\",\"ogit/_type\",\"ogit/name\",\"ogit/_scope\"\n\"timeseries\",\"ogit/Timeseries\",\"tsname\",\"scopeid\"Then
\nWith the token in hand you can run a PUT request against\nhttps://api.hiro.almato.ai/function/csv-connector/timeseries
\n\"ogit/_xid\",\"timestamp\",\"value\"\n\"timeseries\",\"1442235678000\",\"45\"Feedback is provided if the csv file was processed successfully and how many objects of each type were populated.
\nA hard stop occurs incase the reading of a csv file encounters a malformed entry and the error is not recoverable. All items before the error will be processed.
\nIf the error is recoverable, an error message with status \"fail\" will be put onto the result data and the operation will continue.
\nSpecifications of the format of csv data so the CSV connector can parse the data correctly. Parsing is based of\nPythons' csv package.
\nGeneral format
\nThe encoding is UTF-8.
\nThe separator is a comma ,.
Each row must have the same number of values.\nRows must be filled with empty values if necessary.
\nEach value in the CSV must be enclosed in double-quotes ".\nQuotes are escaped via doubling, i.e. "I am ""quoted"".".
The CSV has to have a header row.\nFor vertices these are the ogit attribute names.
\nThe CSV will be transformed using Pythons' csv package with a special dialect called default (unless configured otherwise in header parameter x-connector-csv-dialect).\nThe settings for this dialect are:\nquoting=csv.QUOTE_ALL, skipinitialspace=True, strict=True
Each row below the header specifies a set of values for one entity (vertex, edge, timeseries or attachment).
\nThe strings parsed from each cell can be interpreted as JSON when header parameter x-connector-csv-json-strings is enabled.\nThe values of the CSV will then be enclosed in quotes and run through json.loads(escaped_value).\nIf this is set to true, special characters like \\, ", line-breaks or tabulator characters need to be properly escaped for JSON and remember that you still have to quote the " for CSV as well.
Examples of a CSV cell
\nwithout JSON Strings:
\n\"Hello \"\"friend\"\",\nhow are you? Look at C:\\Users\\Foo\\\"\n
with JSON Strings:
\n\"Hello \\\"\"friend\\\"\",\\nhow are you? Look at C:\\\\Users\\\\Foo\\\\\"\n
Format of the CSV data for vertices
\nRelevant REST endpoints:
\nPOST https://api.hiro.almato.ai/function/csv-connector/vertices\nPATCH https://api.hiro.almato.ai/function/csv-connector/vertices\nPUT https://api.hiro.almato.ai/function/csv-connector/vertices\nDELETE https://api.hiro.almato.ai/function/csv-connector/verticesThe header row contains ogit attribute names.
\nAn empty value "" in a row tells the connector to ignore the attribute specified by that column for this row instead of setting it to an empty value.
A string value of "null" (or whatever is currently configured in header parameter x-connector-csv-null-keyword) in a row will make the connector set the attribute specified by that column explicitly to JSON null.\nThis is needed to remove attributes of a vertex on an update.
Headers that start with xid: and contain ogit/_xid values are resolved to ogit/_id by querying HIRO.
Example:
\nxid:ogit/Automation/originNode would treat all values below as ogit/_xid and resolve them to ogit/_id before using them as attribute ogit/Automation/originNode in the vertices to create or update.
Headers that start with =, + or - denote map value entries (see HIRO attribute specification).\nValues within these columns are directly interpreted as JSON.
Example:
\n\"ogit/_xid\",\"ogit/_type\",\"ogit/MARS/Machine/class\",\"ogit/name\",\"ogit/MARS/Machine/ram\",\"/admin_contact\"\n\"machine4\",\"ogit/MARS/Machine\",\"Linux\",\"machine 4\",\"2G\",\"info@admin.co\"\n\"machine5\",\"ogit/MARS/Machine\",\"Linux\",\"machine 5\",\"4G\",\"contact@admin.co\"\n\"machine6\",\"ogit/MARS/Machine\",\"Linux\",\"machine 5A\",\"8G\",\"info@admin.co\"\n\"machine7\",\"ogit/MARS/Machine\",\"Linux\",\"machine 4A\",\"4G\",\"contact@admin.co\"Format of the CSV data for edges
\nRelevant REST endpoints:
\nPOST https://api.hiro.almato.ai/function/csv-connector/edges\nDELETE https://api.hiro.almato.ai/function/csv-connector/edgesThe CSV has to have three headers:
\nfrom:ogit/_id or from:ogit/_xid
verb The ogit verb for the edge
to:ogit/_id or to:ogit/_xid
Example:
\n\"from:ogit/_xid\",\"verb\",\"to:ogit/_xid\"\n\"crew:NCC-1701-D:picard\",\"ogit/Forum/mentions\",\"crew:NCC-1701-D:data\"Format of the CSV data for timeseries
\nRelevant REST endpoints:
\nPUT https://api.hiro.almato.ai/function/csv-connector/timeseriesThe CSV has to have three headers:
\nogit/_id or ogit/_xid
timestamp for ms since epoch or time for an ISO time
value for the value for this timestamp.
Example:
\n\"ogit/_xid\",\"timestamp\",\"value\"\n\"timeseries:machine4\",\"1442235678000\",\"45\"\n\"timeseries:machine4\",\"1442235944000\",\"43\"\n\"timeseries:machine5\",\"1442235678000\",\"51\"\n\"timeseries:machine5\",\"1442235944000\",\"48\"or two headers:
\nogit/_id or ogit/_xid
items for a JSON list of objects containing the keys (timestamp or time) and value.
Example:
\n\"ogit/_xid\",\"items\"\n\"timeseries:machine4\",\"[{\"\"timestamp\"\": \"\"1442235678000\"\", \"\"value\"\":\"\"45\"\"},{\"\"timestamp\"\": \"\"1442235944000\"\", \"\"value\"\":\"\"51\"\"}]\"\n\"timeseries:machine5\",\"[{\"\"timestamp\"\": \"\"1442235678000\"\", \"\"value\"\":\"\"43\"\"},{\"\"timestamp\"\": \"\"1442235944000\"\", \"\"value\"\":\"\"48\"\"}]\"Format of the CSV data for attachments
\nRelevant REST endpoints:
\nPUT https://api.hiro.almato.ai/function/csv-connector/attachmentsThe CSV needs the following headers:
\nogit/_id or ogit/_xid
One of the following headers is required for the content of the attachment:
\ncontent:text: Content to upload as simple text in UTF-8.
content:base64: Content to upload as base64 encoded binary data.
content:mimetype: (optional) Content-Type of the content.
Example:
\n\"ogit/_xid\",\"content:mimetype\",\"content:text\",\"content:base64\"\n\"attachment:arago:test:0:lorem-ipsum\",\"text/plain\",\"Auch gibt es niemanden, der den Schmerz an sich liebt, sucht oder wünscht, nur, weil er Schmerz ist, es sei denn, es kommt zu zufälligen Umständen, in denen Mühen und Schmerz ihm große Freude bereiten können.\n\nUm ein triviales Beispiel zu nehmen, wer von uns unterzieht sich je anstrengender körperlicher Betätigung, außer um Vorteile daraus zu ziehen? Aber wer hat irgend ein Recht, einen Menschen zu tadeln, der die Entscheidung trifft, eine Freude zu genießen, die keine unangenehmen Folgen hat, oder einen, der Schmerz vermeidet, welcher keine daraus resultierende Freude nach sich zieht? Auch gibt es niemanden, der den Schmerz \",\"\"\n\"attachment:arago:test:1:lorem-ipsum\",\"text/plain\",\"\",\"QXVjaCBnaWJ0IGVzIG5pZW1hbmRlbiwgZGVyIGRlbiBTY2htZXJ6IGFuIHNpY2ggbGllYnQsIHN1\nY2h0IG9kZXIgd8O8bnNjaHQsIG51ciwgd2VpbCBlciBTY2htZXJ6IGlzdCwgZXMgc2VpIGRlbm4s\nIGVzIGtvbW10IHp1IHp1ZsOkbGxpZ2VuIFVtc3TDpG5kZW4sIGluIGRlbmVuIE3DvGhlbiB1bmQg\nU2NobWVyeiBpaG0gZ3Jvw59lIEZyZXVkZSBiZXJlaXRlbiBrw7ZubmVuLgoKVW0gZWluIHRyaXZp\nYWxlcyBCZWlzcGllbCB6dSBuZWhtZW4sIHdlciB2b24gdW5zIHVudGVyemllaHQgc2ljaCBqZSBh\nbnN0cmVuZ2VuZGVyIGvDtnJwZXJsaWNoZXIgQmV0w6R0aWd1bmcsIGF1w59lciB1bSBWb3J0ZWls\nZSBkYXJhdXMgenUgemllaGVuPyBBYmVyIHdlciBoYXQgaXJnZW5kIGVpbiBSZWNodCwgZWluZW4g\nTWVuc2NoZW4genUgdGFkZWxuLCBkZXIgZGllIEVudHNjaGVpZHVuZyB0cmlmZnQsIGVpbmUgRnJl\ndWRlIHp1IGdlbmllw59lbiwgZGllIGtlaW5lIHVuYW5nZW5laG1lbiBGb2xnZW4gaGF0LCBvZGVy\nIGVpbmVuLCBkZXIgU2NobWVyeiB2ZXJtZWlkZXQsIHdlbGNoZXIga2VpbmUgZGFyYXVzIHJlc3Vs\ndGllcmVuZGUgRnJldWRlIG5hY2ggc2ljaCB6aWVodD8gQXVjaCBnaWJ0IGVzIG5pZW1hbmRlbiwg\nZGVyIGRlbiBTY2htZXJ6IAo=\"Simultaneous handling of vertices, issues, edges and/or attachments
\nRelevant REST endpoints:
\nPUT https://api.hiro.almato.ai/function/csv-connector/combinedThe format of the CSV data for vertices applies here, see above.
\nEdges
\nThe format of an edge header is edge:[id]:[attribute]. [id] is an identifier to group edge commands together (most simple case: a number).\nThis [id] must not contain a : since this is used as separator.
edge:[id]:verb: (required) Verb for that edge for the vertex of the current row.
edge:[id]:direction: (\"in\"/\"out\") from the view of the current vertex of that row. \"in\" points towards, \"out\" points away from the current vertex.\nDefault is \"out\" if this column is missing.
One of the following headers is required to find the vertex to connect to:
\nedge:[id]:vertex_id: ogit/_id of the other vertex to connect to.
edge:[id]:vertex_xid: ogit/_xid of the other vertex to connect to.
Attachments
\nThe format of an attachment header is content:[attribute]:
One of the following headers is required for the content of the attachment:
\ncontent:text: Content to upload as simple text in UTF-8.
content:base64: Content to upload as base64 encoded binary data.
content:mimetype: (optional) Content-Type of the content.
Issues
\nThe format of an issue header is issue:[id]:[attribute]. [id] is an identifier to group issue commands together for multiple issues to be created. Some attributes of each issue will be set automatically:
ogit/_type: Will be set to ogit/Automation/AutomationIssue
\nogit/Automation/originNode: Will be set to the ogit/_id of the vertex that has been created on the same row.
\nExample:
\n\"ogit/_xid\",\"ogit/_type\",\"ogit/name\",\"ogit/Forum/username\",\"edge:0:verb\",\"edge:0:direction\",\"edge:0:vertex_xid\",\"edge:1:verb\",\"edge:1:direction\",\"edge:1:vertex_xid\"\n\"crew:NCC-1701-D:picard\",\"ogit/Forum/Profile\",\"Jean-Luc Picard\",\"Picard\",\"ogit/Forum/mentions\",\"out\",\"crew:NCC-1701-D:data\",\"ogit/Forum/mentions\",\"out\",\"crew:NCC-1701-D:worf\"\n\"crew:NCC-1701-D:worf\",\"ogit/Forum/Profile\",\"Worf\",\"Worf\",\"ogit/subscribes\",\"out\",\"crew:NCC-1701-D:picard\",\"\",\"\",\"\"\n\"crew:NCC-1701-D:data\",\"ogit/Forum/Profile\",\"Data\",\"Data\",\"ogit/Forum/mentions\",\"out\",\"crew:NCC-1701-D:worf\",\"ogit/subscribes\",\"in\",\"crew:NCC-1701-D:worf\"A complete API documentation is available under https://api.hiro.almato.ai/function/csv-connector/docs
\n