Gateway Releases
 

The Innovation Gateway requires a significant volume of design and development work to deliver our vision and ambition. To achieve this our teams are continually working on the Gateway delivering major software releases approximately every 4 weeks.

On this page we explain what new developments we are currently working on and list the major releases made since the Innovation Gateway was released in its current format in June 2020. Clicking on an entry provides further detail about the functionality that the particular release delivered.

 

Active development activities

We are actively working on the following features which, once ready, will be made available across one or more releases and reported in the release notes below.

  • Data custodian onboarding to 5 safes forms: We continue to work with data custodians to adopt the more advanced 5 safes data access request (DAR) forms, providing the enhanced DAR functionality and user experience that they offer across an increasing number of datasets.
  • Federated Identity Brokerage: Integrating with other trusted identify providers to provide you with an easier authentication/login experience.
  • 5 safes Data Access Request improvements (researcher): Enabling you to delete pre-submission DARs; Validating your DAR and asking you for confirmation before it is submitted; Enabling you to copy the answers to the questions in an existing DAR form into a new form to request datasets from a different custodian (cloning).
  • 5 safes Data Access Request improvements (custodian): Sending pre-submission messages from researchers to team generic inboxes.
  • Homepage and site navigation redesign: Complete overhaul of the Gateway homepage, site navigation and content in response to your feedback (researchers, data custodians and the public).
  • Search within collections: Enabling you to search within collections in addition to the Gateway's existing search functionality.
 

May has been a busy month for The Gateway. For our second May release we are very excited to reveal to you our new homepage design! It's brighter and fresher, with careful consideration paid to providing you the most information in the clearest possible way.

As if that wasn't enough, this release also contained improvements to the way a user is able to sort their search results, the ability for data custodians to have data access request forms sent to a generic email inbox and we have updated our Ts & Cs to to offer our audience the opportunity to stay in touch.

Our first May release delivered a major step forward for the 5 Safes Data Access Request forms (DAR), with applicants now able to create new DARs quickly by cloning an existing one. That means answers can now be copied from one application form into another. This will see a significant reduction in re-keying and data entry time for completing separate submissions to other custodians using 5 Safes.

This new functionality also paves the way for future developments on the DAR including the ability to amend, renew and extend applications, as well as applying for datasets across multiple custodians with a single form, and question bank.

We have now completed the build of HDR UK's own metadata catalogue and successfully transitioned data on all of the datasets you could previously find on the Gateway into the new catalogue. This improves stability and offers more control to improve the quality and completeness of metadata on the Gateway. It will provide you, and other researchers and innovators, with an improved experience when discovering healthcare data.

For updating or adding additional metadata, the metadata interface and workflow are now fully integrated into the Gateway. This provides a single, consistent user experience for data custodians, with native support for dataset versioning, making it easier for custodians to onboard/update information about their datasets onto the Gateway. And for those of you with your own in-house metadata catalogue, or if you are considering implementing one, our data model also supports interoperability with these. Please contact Alison.Elderfield@hdruk.ac.uk for more information.

Our April 2021 release includes some exciting new functionality - Cohort Discovery - as well as a significant improvement and extension to the dataset discovery filtering functionality within the Gateway.

Cohort Discovery: An innovative and exciting new Gateway feature, Cohort Discovery will allow you to search across many datasets to find those that include groups, or cohorts, of patients with specific characteristics that you require to support your research. For example, you could use the Cohort Discovery functionality to look for datasets that contains patients that don't smoke aged between 18-30 and who live in London. You can access the Cohort Discovery feature within the Gateway here.

Advanced dataset discovery filtering: The HDR UK Improving and Uniting Data teams have been working with data custodians to increase the breadth of the data describing each of the datasets - called metadata - listed on the Gateway.

To allow you to filter your search results during dataset discovery using this broader set of metadata, we have completely reengineered the way the filters work to use this broader set of metadata to create a much more compelling experience for you.

  • New filter categories: We have introduced new filter categories (and sub-categories) for Coverage; Provenance; Access; Format and standards; Data utility; and Technical Metadata. In addition to the existing filter categories of Keywords; Phenotype; Geographic coverage these provide you with a more powerful and flexible way to focus on the datasets that meet your research needs.
  • Advanced filtering functionality: In extending the filters we have also integrated search functionality into the filters that contain larger numbers of filter options e.g. the Keywords; Phenotype and Publisher filters, helping you find relevant filter options.

Data Access Requests: You can now delete Data Access Requests that are in a pre-submission state, enabling you to remove those draft access requests that are no longer needed. We have also introduced system validation of your DAR and a confirmation request from you before it is submitted. This will help you to avoid submitting applications prematurely or by mistake.

User support documentation: We have expanded, re-categorised and simplified the support pages for the Gateway to make it easier for you to find information to answer any questions you have about Gateway functionality or how you complete a task.

Performance and stability improvements

We have made the following performance and stability improvements as part of this release. These may not be obvious to all users.

  • Improving support for Internet Explorer 11.
  • Improving performance and stability of the metadata utility calculations.
  • Making enhancements to the Gateway database schema

Our March 2021 release continued to add further functionality to the 5 Safes Data Access Request forms, greater flexibility to Gateway collections and enhancements to the Gateway APIs.

Data Access Requests: For a data access request you have yet to submit (pre-submission DAR), you can export the questions and any answers you have completed into an email to share or print. You can also now delete any documents you've uploaded to a pre-submission DAR. Finally, we have added a scan indicator to let you know when documents you've uploaded are being checked for viruses.

Collections: You can now make your own Collections public for other Gateway users to see. At the same time we have also improved the look-and-feel of the collection cards and the keyword functionality making it easier for all users to discover and explore collections on the Gateway.

Searchable collections: When you run a search this now includes the Collections published on the website ensuring any Gateway resources that are relevant to your search are returned.

Gateway APIs: We have added further functionality to the Gateway's APIs making it easier for Custodian's access management systems to interoperate with the Gateway.

Performance and stability improvements

We have made the following performance and stability improvements as part of this release. These may not be obvious to all users.

  • Notifying you at login about about new mandatory user profile fields
  • Display of linked gateway resources has been improved to ensure they are listed for both linked resources.
  • Website improvements for mobile/tablet users.

Our February 2021 release focused on adding further functionality to the 5 Safes Data Access Request forms, onboarding a new data custodian to using the form and improvements to papers and collections.

Data Access Requests: As you complete a data access request an application question counter will now show how far you are through the questions for each section. The forms have also been extended so you can now add multiple people, organisations, and funders to a data access request. And finally, you can no longer submit a data access request for an old version of a dataset.

ISARIC now using 5 Safes forms: ISARIC are now using the enhanced 5 Safes data access forms for the data sets they have listed on the Gateway.

Papers: You are now able record multiple Digital Object Identifiers (DOIs), URLs or PDF links for each paper you add to the Gateway.

Collections: We have improved the layouts to collections and made featured collections more prominent to make them easier to identify.

Performance and stability improvements

We have made the following performance and stability improvements as part of this release. These may not be obvious to all users.

  • Where a request results in a large number of results we have now split these over multiple pages to keep page loading times to a minimum.
  • Duplicating abstract for Papers has been removed.
  • Website improvements for mobile/tablet users.
  • Accessibility improvements.
  • Ensuring IE/Safari support for user registration

Following a slightly longer sprint due to Christmas and New Year, our January 2021 release focused on adding further functionality to the 5 Safes Data Access Request forms along with some additional improvements in response to your feedback.

Faster results filtering by custodian: You can now quickly filter your dataset search results for a single custodian by clicking on the custodian's name within the search results which is beneath the dataset name.

User feedback buttons: We encourage user feedback and you can now reports bugs and suggest feedback from the footer of every page within the Gateway.

Data access requests: Where you don't complete an answer to a question, these are now highlighted to you making it easier to ensure you complete your access requests correctly.

Collection navigation improvements: We have removed the All tab from collections making it easier for you to find the resources that you need within each collection.

Performance and stability improvements

We have made the following performance and stability improvements as part of this release. These may not be obvious to all users.

  • Ensuring IE/Safari support for user registration.
  • Migrating 190 Courses to support the new courses entity.

We have made the following performance and stability improvements as part of this release. These may not be obvious to all users.

  • Support for IE user registration.
  • Fixing a small number of redirect links

We have made the following performance and stability improvements as part of this release. These may not be obvious to all users.

  • Update to metadata quality to use weighted scoring.

We have made the following performance and stability improvements as part of this release. These may not be obvious to all users.

  • Improvements to metadata catalogue caching performance and stability.
  • Fixed multiple minor issues with 5 safes data access request forms.
  • Fixed issue with National Core Studies data access request schema.

Our November 2020 release - the first of Milestone 2 - focused on adding further functionality to the 5Safes data access request forms, onboarding a number of custodians to use the new form along with a number of other improvements in response to your feedback.

National Core Studies & Early Adopters* now using 5 Safes forms: NCS's and Early Adopter organisations are now using the enhanced 5 Safes data access forms for the data sets they have listed on the Gateway.

Persistent dataset IDs: Datasets within the Gateway now have persistent IDs rather than the ID changing with each new version of the same dataset as provided by the metadata catalogue supplier. This will make it much easier for you to reference datasets in your resources helping to avoid using links that are out of date.

Dataset v2 updates: Dataset metadata has been transitioned to the new, more advanced, version 2 schema. This provides the foundation for you to undertake more advanced searches and results filtering in the future.


* National Core Studies & Early Adopters are: NHS Digital; Office for National Statistics; NHS Scotland; SAIL Databank; NI Honest Broker Service; INSIGHT HDR UK Hub.

We have made the following performance and stability improvements as part of this release. These may not be obvious to all users.

  • Fixed issue where search figures were not showing correctly on the dashboard page.

Our early November 2020 release focused on 5 Safes Data Access Requests (DARs), Data Custodian workflows and adding educational materials to the Gateway along with a number of other improvements in response to your feedback.

5 Safes DARs: In this release we have deployed a new standard, yet configurable, DAR form with the objective of making it easier for you to submit compliant DARs that were "right first time" enabling data custodians to effectively process them without having to seek clarification from requestors. We wanted to build this around a recognised and respected framework for considering access to confidential or sensitive data and an obvious choice was the Office of National Statistics' (ONS) Five Safes model which considers each request across five dimensions:

  • Safe projects
  • Safe People
  • Safe settings
  • Safe data
  • Safe outputs

Working with custodians we have defined the common, and custodian specific questions and guidance that would be required under each dimension for them to be able to process a DAR. We are now working with an initial set of custodians for them to adopt the new 5 safes DARs.

Educational materials (Courses): You can now add, edit and remove educational materials to the Gateway.

Data custodian workflows: You can now create multiple workflows and customise them by assigning people from your team to each step of a workflow. Every time a new DAR is submitted, you can also choose which of your workflow processes that you would like to assign to each DAR.

Uploading and downloading files: You can now upload files to the Gateway when adding resources to the Gateway. Any user is then able to download them. files are virus scanned as part of the upload process to protect the Gateway and other users' devices.

We have made the following performance and stability improvements as part of this release. These may not be obvious to all users.

  • Fixed an issue where submitting a request to access the HDR UK Papers project fails.

We have made the following performance and stability improvements as part of this release. These may not be obvious to all users.

  • Fixed issue where technical metadata details were blank for some datasets.
  • Fixed issue where datasets with over 100 elements of technical metadata were only showing the first 100 elements to users.
  • Fixed minor issue with edit button labelling on Gateway entities.

Our October 2020 release focused on a new design for Gateway Collections and the introduction of Collection categories. Further work has also taken place on Data Access Requests (DARs) as well as the release of a new dashboard reporting Data Access Requests (DARs). We have also progressed a number of other improvements in response to your feedback.

Collections & Collection categories: In response to your feedback we have refreshed the design for Collections on the Gateway. This will make it easier for you to discover collections and their content. We have also added functionality to group Collections within a Category. This will allow Collections with a common theme to be bundled together making it easier for you to see related Collections all in one location on the Gateway.

DAR Activity dashboard: A dashboard is now available on the Gateway where you can explore at-a-glance information about the Gateway including: monthly statistics; the datasets that have received the most access requests; top searches; and unmet demand i.e. searches that yielded no results.

We have made the following performance and stability improvements as part of this release. These may not be obvious to all users.

  • Fixed issue with Single Sign On (SSO) for some users.

Our August 2020 release focused on improving search and filtering functionality, making Gateway statistics available through a new analytics dashboard and providing additional functionality to help manage Collections. We have also progressed a number of other improvements in response to your feedback.

Search improvements & Advanced Filtering: We have improved the performance of the Gateway's dataset search functionality so that your search results are more accurate. In addition, advanced filtering functionality provides you with more options to reduce your search results based on various criteria to help you discover the resources that best meet your needs.

Analytics dashboard: We have released a dashboard page listing a collection of statistics, metrics and analytics to provide you with an overview of the Gateway's data and performance.

Archiving Collections: You can now archive and unarchive Collections that you have created on the Gateway.

Bulk load of Papers: 400 papers have been loaded into the Gateway significantly improving the richness of resources available to you as a user.

Unsubscribing from Latest news emails: You can now unsubscribe from the latest news emails using the link at the bottom of the email message.

We have made the following performance and stability improvements as part of this release. These may not be obvious to all users.

  • Fixed an issue where a blank screen appeared when trying to upload a project.
  • Fixed an issue with the metrics displayed on the homepage.
  • Fixed an issue where part of the homepage does not display correctly on some browsers.

Our July 2020 release focused on the first version of our Data Access Request (DAR) form as well as introducing Collections to the Gateway.

Data Access Requests (DARs): You can now submit an enquiry to data custodians to request access to one of their datasets listed on the Gateway.

Collections: You can now use Collections to group Gateway resources together for future reference and quick access. These could be based on a theme or just a group of favourite resources from across the Gateway.

Our July 2020 release focused on the first version of our Data Access Request (DAR) form as well as introducing Collections to the Gateway.

Data Access Requests (DARs): You can now submit an enquiry to data custodians to request access to one of their datasets listed on the Gateway.

Collections: You can now use Collections to group Gateway resources together for future reference and quick access. These could be based on a theme or just a group of favourite resources from across the Gateway.

We have made the following performance and stability improvements as part of this release. These may not be obvious to all users.

  • Provided functionality to enable users to opt-out of Google Analytics cookies
  • Transferred the domain to https://healthdatagateway.org
  • Updated our About page to list the browsers we support
  • Fixed an issue when logging in with LinkedIn
  • Fixed an issue with word-wrapping on the landing page

Our 1 June 2020 release marks the start of the next stage in our journey to deliver a tool to make it easier for you - researchers and innovators - to discover and request access to UK Health datasets.

In this first release we have focused on providing you with key functionality to enable dataset discovery. You can now search and sort across an initial series of UK health datasets from a number of UK data custodians helping you to discover resources to help you with your research or business. We have also released functionality for you to record and share the research projects that you have worked on, or are working on, as well as the tools you use to conduct your research.