Blog Layout
London Museum - CMS/DAMS integration
4 August 2020
Background
The London Museum
had a requirement to better manage its collections and non-collections digital assets. These had been managed in different systems which were not integrated, resulting in duplication of effort and data about the same asset which was not synchronised. Any solution would need to integrate with its collections management system (CMS), the system of record for descriptive and contextual information about its extensive object collections.
The museum decided to procure Portfolio DAMS from Extensis, who worked in partnership with Knowledge Integration - who provide middleware that can act as broker between different information systems, enabling seamless transfer of data.
Alex Bromley Consulting was commissioned to project manage the integration between the three systems.
The work
The first act was to draw up a document outlining the principles upon which the integration would work. This was a vital first step to ensure all parties knew the scope of work. Issues included in the Data Management Policy were:
- Number and type of records to be shared between the three systems.
- Which application was the system of record for which data type.
- Initial scope of integration between the three systems, i.e. direction and frequency of ingest/sharing.
- What of the various file types were to be ingested by the DAMS.
- The metadata that would provide the link between the three systems.
- Nature and extent of data transformation required.
- Statement on potential use of taxonomies.
The DAMS team at the museum developed a metadata schema for Portfolio, covering their requirements for both collections and non-collections assets. There also needed to be a field in the CMS to act as a ‘flag’ to indicate whether a record was ‘DAMS Ready’.
Using this as the basis, a mapping was made between their schema and that of the collections management system, taking into account field type and definitions. This was given to Knowledge Integration who used the CIIM to pull the relevant fields of the chosen records at a set frequency, then push them into Portfolio. User permissions in Portfolio enforced data integrity, ensuring this information was not edited by users of the DAMS.
After consultation with Knowledge Integration, a decision was made on the common element between the two data sets that would act as the ‘join’. In this case it was the asset file name and path (the museum had a mixture of filenaming conventions developed over the last twenty years so using the filename alone would not have worked).
A vital piece of information that DAMS users would require was copyright and credit line information about collections images, currently stored in the CMS. Rights data in the collections management system had been partially cleaned after an upgrade to the application in 2016 and any rules had to take into account the differing levels of quality and detail of rights data available. Where data had been cleaned a detailed series of check boxes indicated what were the permitted uses.
As a baseline, it was agreed that any rights data that had not been ‘cleaned’ in the home system would not be pulled into DAMS and users were directed to seek help from the Picture Library. Where data had been cleaned (which had been conveniently flagged in a check box) then I developed transformation rules which took the permitted uses (stored in a Y/N check box) and turned them into a bulleted list of permitted uses, topped and tailed with user friendly text.
Knowledge Integration utilised their Collections Information Integration Middleware (or CIIM) to do this transformation and push from collections management system to Portfolio.
Outcomes
The project achieved a minimum viable product for the museum, enabling them for the first time to utilise a digital asset management system which shared data with another application containing related information. Users of the DAMS are now able to search both collections and non-collections images, view descriptive and contextual metadata, as well as vitally important rights data outlining permitted uses.
For the moment, the museum will use a manual system for moving images from the network and placing them into Portfolio, based on the existing set of 90,000 object records and associated images which has been published online. Digitisation procedures for collections and non-collections images continue to utilise different approaches but over time these will be integrated into Portfolio, with that application serving as the system of record for all digital assets in the museum. In this future scenario, the CIIM would synchronise data two ways - CMS data to the DAMS, and media and media metadata from DAMS to CMS.
Background BFI Replay is a new streaming platform, only available in public libraries, showcasing over 10,000 digitised titles from the BFI National Archive , as well as the holdings of the Regions and Nations Archives around the UK. Digitised as part of the Heritage 2022 programme (H22), these are derived from 100,000 titles previously only held on videotape, but now digitised for preservation and access. In addition, metadata and descriptions, content and trigger warnings as well as overarching ‘Collection’ pages put the titles in their historical and social context. The work Alex Bromley Consulting was engaged to project manage the final stages of the ‘Access’ strand of H22, working with colleagues in the National Archive, Digital and Rights teams to take the platform up to the point of ‘soft launch’. Subsequently, the consultancy worked to provide closedown and evaluation activities for the H22 programme, ensuring smooth transfer of the processes and product support to ‘business as usual’. In addition, BFI Library and H22 colleagues were supported in rolling out the product across the UK to over 2000 libraries, through a series of launch events across the country. Key Activities undertaken throughout the programme: Re-setting the ‘Access’ Project Plan so all teams were aware of targets, quality thresholds and roles and responsibilities. Chairing weekly stand-up meetings for the ‘Replay development’ team. Facilitating and documenting discussions between videotape engineers in the Archive, Live Services team and Curators to ensure quality issues and thresholds were commonly understood (videotape by its nature can be ‘glitchy’). Liaised with Regions and Nations Archives for the delivery of metadata and progress updates on Replay development. Progress tracking for ingest and publication to the platform. Managing Closedown and Evaluation activities including final report submitted to Lottery. Reporting to Project Delivery and Programme Management Groups on progress, issues and risk. Coordinating user evaluation activities. Onboarding new staff dedicated to Replay platform development and promotion. Managing research and development into speech-to-text options for the platform. BYOD Workshops with libraries hosting Replay. Since the end of H22, Alex Bromley Consulting has produced the following process support documentation to ensure the platform is sustainable, so that it can flourish and continue to ingest content and attract new libraries to sign up: ‘Business as Usual’ roles and responsibilities – analysing the ‘project’ roles and comparing, documenting and gaining agreement on resources required to sustain the platform going forward. BFI Replay Policy – documenting the key decisions made during the content and technical development that BFI staff need to be aware of when adding new content, signing up new libraries, or responding to enquiries from the public or content/rights providers. BFI Replay BYOD Policy – outlining the key points libraries need to be aware of when allowing users to ‘Bring Your Own Device’. Procedural documentation to support workflow for ingest and publication, reporting and takedown as well as sign up and support for new host-libraries. Outcomes for the client The BFI Replay product was delivered to time and with the target number of titles and Collections successfully uploaded. All project documentation and evaluation products were maintained and archived for closedown, including Lessons Learned session for future projects. Project stand-ups were chaired and documented to allow rapid development of the ‘ingest pipeline’, the product and facilitate information sharing with colleagues throughout the BFI and beyond. Soft and full launch successfully achieved with over 2000 library sites now signed up to the host the platform, achieving the BFI’s stated aim of widening the audience for the archive’s work through the product. All Policies and Processes documented for the BFI, ensuring new and existing staff are up to speed with agreements and protocols agreed during the H22 programme. Platform successfully delivered into a ‘business as usual’ environment.
Background The Science Museum Group (SMG) have presented an aggregated collections online offering from all their sites since the early 2000s. However, while the scale of digitisation and the amount of information available online was impressive (368,688 object records online, 195,798 with an image), evaluation by SMG had revealed that engagement with the content could sometimes be restricted. Four issues were identified: Due to the nature of how ‘groups’ and ‘multi-part objects’ were catalogued in back-end systems it was difficult for users to navigate between related objects. Media associated with an object was sometimes missing or associated with different record in the object hierarchy. A large-scale digitisation project at Blythe House resulted in large numbers of images of part records that could not be published without the further contextualisation of the object. Issues with how media and object records were extracted from backend systems and aggregated in the CIIM middleware could be improved to create internal efficiencies as well as better experience for end users. It was decided by SMG to wrap up these improvements into a new programme of work -Collections Online 2.0 - to develop and improve the offering for users of their collection search site and make the overall data management ecosystem more resilient and agile. The work Alex Bromley Consulting was engaged in two phases to conduct firstly a discovery phase, and then an implementation phase. The following summary outlines the nature and scope of works to achieve the desired business benefits: Phase 1: Discovery Phase Review of existing cataloguing rules, CIIM extraction routine and Snap Logic pipeline documentations. Interviews with stakeholders to explore requirements, issues and risks. Interviews and survey carried out with peer museums to share knowledge and potential solutions. Workshop new ‘data model’ for groups and multi-part objects with curators, collections managers and digital team. Report including costed recommendations under three headings: ‘Closing the image gap’ – getting more images online. Exploration and Visualisation – displaying groups and multi-part objects. Search and Discovery – data cleaning recommendations to improve resource discovery. Phase 2: Implementation Phase Creation of project plan including 12-month timeline Refining ‘data model’ recommendations and gaining approval from senior stakeholders. Chairing project team meetings, tracking progress, issues and monthly reporting to One Collection Board Project manage upgrade to iBase DAMS. Project manage upgrade to Knowledge Integration’s CIIM middleware. Creation of data cleaning plan for post-project implementation Outcomes The Collections Online 2.0 project will provide the Science Museum Group with the tools, processes and data to better display and interpret their collections online. New web pages for ‘groups’ and ‘multi-part’ objects now enable users to explore the inter-relationships between museum objects and their constituent parts, and the number of associated media available has increased. Further data cleaning in the collections management system Mimsy XG, as well as the DAMS, will improve users’ experience as these flow through the automated extraction routines. In summary, the following business benefits were achieved through the lifetime of the two phases: Improvements to the image ingest pipeline – a SnapLogic solution which integrated the collections management system (Mimsy XG) and DAMS (iBase) A new ‘data model’ for groups and multi-part objects, as well as a data cleaning plan to implement these changes. Improvements to the way the CIIM middleware extracted data from the DAMS and collections management system, enabling: More data to be displayed online. More accurate rights data for internal and external audiences. Intelligent display of multi-part and groups of objects, according to a set of pre-defined rules. Design and development of new ‘ group ’ and ‘ multi-part object ’ web pages integrated with improved back-end systems.
Background The BFI had procured ArtifaxEvent venue management system to streamline programming, scheduling and venue hire activities at BFI Southbank and Festivals. The implementation project was already underway, and improvements had been made in several areas of the business by a small but dedicated project team. After an internal promotion for the existing project manager, Alex Bromley Consulting was commissioned to manage the remaining period of the project. In addition to managing the work of the supplier and project team to see the system delivered on time, the BFI requested a full review of the business benefits achieved. The project board also asked for recommendations for how the system could be embedded into the organisation to sustain the investment, including further integration with related information systems such as finance, HR and CRM. The work Two strands of work were required for this commission: the resetting and monitoring of the original project, as well as the research and analysis required for end of project review. Initially, the existing project plan was reviewed, updated and a revised list of minimum requirements agreed with Artifax. The project board reconvened to approve the plan, which included a series of two-week sprints over six months with the supplier to develop ArtifaxEvent (back end) and ArtifaxAgora (public facing front end). The priority was to work on features which were most urgently required for upcoming phases of the London Film Festival, such as submission and selection of films. A project team was convened, consisting of super users around the organisation, who would feed into developing and testing work during the project phase. After sign-off, the plan was they would then transform into a User Group, to champion use of ArtifaxEvent in the organisation. As a proof of concept, integration with one other information system was planned and achieved using the ArtifaxEvent’s API, the BFI’s collections management system – CID. In parallel with day-to-day project management activities, an end of project review was devised and written, based around the following headings: History and outline of original project’s aims Review of the business case Review of project team, structure and supplier Review of project management strategies and controls Review of Products Follow on recommendations including costed options appraisals Lessons Learned To achieve this, the following tools and techniques were used: Review of project documentation Review of original business case Interviews of key stakeholders and teams Questionnaire to supplier User experience survey for BFI staff Collation of statistics relating to venue hire services before and after implementation Review of business procedures before and after implementation Review of Issues List Outcomes The project achieved the implementation of a centralised scheduling and programme management system which has resulted in the retirement of most of the existing systems and development of more efficient processes for managing information. Where existing systems have been retained for planning purposes, these have been integrated with ArtifaxEvent using upload tools to ensure all data is stored centrally for reporting and long-term preservation. As a result, the BFI now has greater control over its information resources in these areas with business benefits summarised as follows: Providing a system of record and management for Southbank Cinemas, Venue Hire, London Film Festival and Flare. Data stored in a structured, centralised system, more suited to reporting and integrating with other applications. Integration with collections information database. Financial benefits, particularly in terms of Venue Hire activities. Some tasks automated for the first time, including transfer of Festivals submission, selection and accreditation information into central database. Improved records management for Venue Hire activities. Organisational learning about information systems development.
Having a clear understanding of copyright law is vital for information professionals if they are to efficiently and legally manage their collections or assets. What is also important is how this information is presented in a collections management system or DAM system. There may be complex rights issues with some assets, especially born digital assets like film and published works. Here are my top ten tips for a well-managed copyright clearance procedure and data management policy: 1. Stop the problem getting worse. Ensure your current procedures, whether formal or informal, include a step by which you gain the maximum information about rights inherent in the asset right at the beginning. 2. Assess the scale of the 'backlog' problem . You may have been collecting assets for years, resulting in a considerable body of works about which you have no rights information. 3. Consider and document where your potential sources of rights information might be . These could be your collections or asset management system, your corporate archive or even emails. 4. Run a pilot. For each category, take a selection of records and work out how long it will take to review all your information sources. Document the results, including any changes that may be needed to your collections or asset management system. 5. Speak to colleagues about how they currently use your collections or assets so you can develop an organisational approach to rights clearance and/or the use of the exceptions to copyright. 6. Speak to your software vendor about the results of the pilot and your ambitions for rights management in the future. 7. Write a report and present to senior management outlining how long the work might take and what the cost will be (including any software changes). 8. Assuming you get approval, prioritise the work. You should concentrate on works that you know to be in copyright and for which a licence may already exist. 9. If the project is going to take a long time, it may be necessary to run data updates on your collections management or asset system, indicating to users which records have been reviewed and are available for use, and which are still under review. 10. Create a Retrospective Rights Clearance procedure . This will ensure that all staff have a clear understanding of how they should go about rights clearance and use in the future.