README

Set of static assets used (mainly) for ARCHE data preprocessing and ARCHE information pages:

• URI normalization rules used by the ACDH.
  (stored in AcdhArcheAssets/uriNormRules.json)
• Description of input data formats accepted by the ARCHE.
  (stored in AcdhArcheAssets/formats.json)

The repository provides also Python 3 and PHP bindings for accessing those assets.

Installation & usage

Python

• Install using pip3:

  pip3 install acdh-arche-assets

• Use with

  from AcdhArcheAssets.uri_norm_rules import get_rules, get_normalized_uri, get_norm_id
  print(f"{get_rules()}")
  
  wrong_id = "http://sws.geonames.org/1232324343/linz.html"
  
  good_id = get_normalized_uri(wrong_id)
  print(good_id)
  # "https://sws.geonames.org/1232324343/"
  
  # extract ID from URL
  norm_id = get_norm_id("http://sws.geonames.org/1232324343/linz.html")
  print(norm_id)
  # "1232324343"
  
  
  from AcdhArcheAssets.file_formats import get_formats, get_by_mtype, get_by_extension
  
  formats = get_formats()
  matching_mapping = get_by_mtype('image/png')
  matching_mapping = get_by_extension('png')

PHP

• Install using the composer:

  composer require acdh-oeaw/arche-assets

• Use with

  require_once 'vendor/autoload.php';
  
  print_r(acdhOeaw\UriNormRules::getRules());
  print_r(acdhOeaw\UriNormRules::getRules(['viaf', 'gnd']));
  
  print_r(acdhOeaw\ArcheFileFormats::getAll());
  print_r(acdhOeaw\ArcheFileFormats::getByMime('application/json'));
  print_r(acdhOeaw\ArcheFileFormats::getByExtension('json'));

Description of assets

URI normalization rules

Each rule consists of five properties:

• name: a rule name
• match: a regular expression matching rule's URI namespace
• replace: a regex-replace expression transforming an URI in a rule's namespace into its ACDH-canonical form
• resolve: a regex-replace expression transforming an URI in a rule's namespace into an URL fetching an RDF data
• format: an RDF serialization format to be requested while resolving the URL produced using the resolve field

Formats

A curated and growing list of file extensions. For each file extension mappings to the respective ARCHE Resource Type Category (stored in :hasCategory) and Media Type (MIME type) (stored in :hasFormat) are given. The indicated Media Type should only be used as a fallback; it is best practice to rely on automated Media Type detection based on file signatures.

Further information is provided as well.

• fileExtension: File extension to be mapped.
• name: Name(s) the format is known
• archeCategory: The corresponding URI of the ARCHE Resource Type Category Vocabulary
• dataType: A broad category to group formats in; mainly intended for visualisation purposes.
• pronomID: ID(s) assigned by PRONOM
• mimeType: Official Media Type(s) (formerly known as MIME types) registered at IANA.
• informalMimeType: Other MIME types kown for the format
• magicNumber: A constant numerical or text value used to identify a file format, e.g. Wikipedia list of file signatures
• ianaTemplate: Link to template at IANA
• reference: Link(s) to format specifications referenced by IANA and others
• longTerm: Indicates if a format is suitable for long-term preservation.
  Possible values and their meaning
  • yes - long-term format
  • no - not suitable, another format should be used
  • restricted - can be used for long-term preservation in some cases (see comment)
  • unsure - status remains to be evaluated
• archeDocs: Link to a place with more information for the format.
• comment: Any other noteworthy information not stated elsewhere.

Developement (Python)

install needed developement packages pip install requirements_dev.txt

linting, tests and testcoverage

• to run the test: tox
• check coverage and create report: coverage run setup.py test and coverage html
• check linting flake8