A deeper dive into repository assessment: CoreTrustSeal + FAIRenabling capability-maturity
The FAIRsFAIR project has mapped the CoreTrustSeal requirements to the FAIR principles, FAIRsFAIR metrics, and RDA FAIR Data Indicators and created an approach, the CoreTrustSeal+FAIR Capability-Maturity approach, to support repository assessment against the mapped concepts. We at the Finnish Social Science Data Archive (FSD) tested the CoreTrustSeal+FAIR Capability-Maturity approach as a part of our work in EOSC-Nordic work package 4 on FAIR data. The Capability-Maturity approach is currently an informal assessment that repositories can undertake to evaluate their status as a trusted digital repository and FAIRenabling practices.
The assessment revealed things to improve
We went through all the 16 CoreTrustSeal requirements and assessed our practices against each requirement, metric, and indicator. The work involved assigning each requirement a capability-maturity tier level from these five tiers:
1. Initial
2. Managed
3. Defined
4. Quantitatively managed, and
5. Optimised.
In addition, we reviewed the evidence available to support the selected tier, set a target tier for the following review, and assessed whether we covered the FAIR indicators and metrics mapped to each requirement.
In our assessment, two requirements, R05 Organisational Infrastructure and R10 Preservation were at tier 4, and FSD achieved the third tier, “defined,” on most requirements. With a median of 3, FSD is doing quite well, as FAIRsFAIR M4.3 points out that “the current recommendation is that a capability level of 2. Managed across all Requirements should be sufficient to demonstrate CoreTrustSeal compliance and FAIRenabling” and that “[l]evels of 3 and above … are highly desirable”.
However, there is always room for improvement. We assigned tier 2 to three requirements, R03 Continuity of Access, R08 Appraisal, and identification under R13 Discovery & Identification. We set tier 3 as our target for these requirements before our CoreTrustSeal renewal application in 2023. In addition, we set the target tier of 4 or above for four other requirements by the next assessment.
The assessment was a useful exercise that allowed us to identify several aspects to improve our practices and take stock of additional evidence to include in our next CoreTrustSeal application. We also believe we could provide helpful feedback for improving the approach.
Some expertise is required
We found that a good way to conduct the assessment is to bring together a small group of experts on various repository aspects (e.g., administrative, archival services, technical) and work on a table of the requirements by assessing the capability-maturity level, target, and evidence. The process took us two-afternoon meetings and some desk research.
The assessment seems useful to more experienced repositories that have already obtained certification and performed automated FAIR evaluations of their metadata records. The assessment is significantly facilitated by having prior knowledge of CoreTrustSeal requirements, FAIR principles, FAIRsFAIR metrics, and RDA indicators.
The capability-maturity approach appears to fall somewhere between a full CoreTrustSeal self-assessment and FAIR evaluations. We found the idea of maturity levels promising, although we had some challenges finding the correct one for some requirements. The approach brings in some specificity and concreteness to the more general requirements of the CoreTrustSeal.
We noticed that in some cases, the FAIR principles and metrics do not match very well with the CoreTrustSeal requirements, or they do so only partially. This is no fault of the mapping itself but demonstrates that the two have different use cases. They cannot be perfectly aligned but are nonetheless complementary. The emerging capability-maturity approach is a welcome addition to the tools available to assess the FAIRness and trustworthiness of a digital repository.
For more information, please visit:
– FAIRsFAIR project
– M4.3 CoreTrustSeal+FAIRenabling, Capability and Maturity
– CoreTrustSeal
Author: Henri Ala-Lahti, Information Specialist, Finnish Social Science Data Archive and member of EOSC-Nordic work package 4.