11th EMODnet Technical Working Group Meeting

Spring 2022

Meeting Venue: InnovOcean, Oostende and remote meeting – Zoom

Dates: Tuesday 26^th April 2022 (09:30-17:00*), Wednesday 27^th April 2022 (09:30-12:30 Join Session*) (All times are in Central European Summer Time, CEST)

*Joint session with EMODnet Steering Committee (SC), Technical Working Group (TWG) and EC Marine Knowledge Expert Group (MKEG)

Meeting Chairs: Conor Delaney (EMODnet Secretariat) and Zoi Konstantinou (EC, DG MARE)

List of Participants: See Annex I below

Actions list – To be tracked in JIRA

Meeting Minutes

Day 1, Tuesday 26 April 2022

1. Welcome & introduction (Zoi Konstantinou, EC DG MARE; Conor Delaney, EMODnet Secretariat) [1]

Opening

Conor Delaney (EMODnet Secretariat) welcomes all participants and opens the meeting. He indicates the importance of centralisation in order to serve the digital ocean and twin earth initiatives. An overview of the agenda is given.

Zoi Konstantinou (EC, DG MARE) introduces the meeting, indicating there has been a lot of technical work done to achieve the centralisation and more will be required. She thanks the effort done by the consortium. She also welcomes Carlos Cerezo (DG Mare) & Carlos Belladron (DG Mare)

2. EMODnet Centralisation Static Content Update (Tim Collart, Conor Delaney, Francis Strobbe, EMODnet Secretariat; Joana Beja, VLIZ) - see ppt Centralisation static content update

Static content progress for each lot

Cécile Nys (EMODnet Secretariat) gives an overview of the work done on the static content. She has created an overview of all the pages in each portal. Using this overview, she mapped overlapping content onto the Central Portal site map, which significantly reduced the number of pages.

A first iteration of the portal specific content pages has been created on the development central portal, and portal could provide feedback

• She has received feedback from Bathymetry Geology Biology and SBH

• HA, Physics and Chemistry still has to provide feedback on their thematic portal section

ACTION 1: HA, Physics and Chemistry still has to provide feedback on their thematic portal section.

Feedback from Bathymetry has been processed and the page has been updated. Cécile Nys (EMODnet Secretariat) demonstrates how the page currently looks. on https://emodnet.development.ec.europa.eu/en/bathymetry-demo-2

In addition to portal specific pages, thematic portal content is also hosted in common pages

• publications/reports tagged by portal
• helpdesk
• Training and tutorial: new page that will be created for gather more content of portals (Biology MOOC and HA guidelines etc.)
• Video library exists but not a photo gallery at this point.

Francis Strobbe (EMODnet Secretariat) indicates that the main menu structure is being reviewed, and a new proposal is on https://emodnet.development.ec.europa.eu/, and was demonstrated in the meeting

ACTION 2: A Jira ticket will be created to allow the thematic portals to review the menu structure EM-561 - Feedback on Main Menu on EMODnet CP website To Do

Francis Strobbe (EMODnet Secretariat) shows the current layout of the Home page with the graphic elements

Question from Graeme Duncan (EMODnet SBH): He likes that there is a shared reports section, but he would like to know if this will also host guidance documents and documents with Digital Object Identifiers (DOI). Francis Strobbe (EMODnet Secretariat) indicates we can deal with reports that have DOIs by creating a landing page from which the PDF document can be downloaded.

ACTION 3: Francis Strobbe (EMODnet Secretariat) to look into creating a landing page for reports/documents with DOI's

Conor Delaney (EMODnet Secretariat) indicates there will be an ongoing review of all the static content, with a first round by 01 May 2022. This includes:

• Web pages in the portal
• Metadata of each of the datasets (title, description, etc.)
• Descriptions in the map viewer.

Comment of Dick Schaap (MARIS): He likes the approach of Cécile Nys (EMODnet Secretariat) of how thematic portal pages were mapped into the new central portal. However, he is still missing some of the EMODnet Bathymetry services in the common pages. He also indicates that he would like to add some static content to the pages of their services, in order to personalise them to the specific thematic. He also invites the other portals to do this.

Conor Delaney (EMODnet Secretariat) indicates the need to define the typical users in order to design and review the new central portal content. He indicates that given we are on the ec.europa.eu domain, we should focus on the lowest common denominator user: the general public. In the past the thematic portals were more focused on expert users, however on the new central portal, we should focus on general public users on the main web pages, but provide more detailed information as they explore further in the web site.

Graeme Duncan (EMODnet SBH) indicates it would be good to do some testing of the web site that it meets the requirements of the users.

Next steps for static content review

Carlos Cerezo (DG Mare) asks to provide the static content as soon as possible, so the commission can review it. Cécile Nys (EMODnet Secretariat) indicates she will have the Bathymetry content data ready by 06 May 2022. She has to integrate the feedback for Geology Biology and SBH, after which it will be ready for review. Carlos Cerezo (DG Mare) indicates that we agreed to publish all the static content at one time to the production, once everything has been reviewed in the development environment.

Carlos Cerezo (DG Mare) ask if all portals of chemistry (the different domains) will be migrated. Chemistry indicated that indeed all content was included in the review.

Training updates

Carlos Cerezo (DG Mare) There is need for coordination of different editors and publishers to work in the same environment/content manager. We need to establish a procedure/rules for the content curation. How to work in the different environment (development/production).

There will be a training organised by DG COMM to inform all editors/publishers of the requirements in publishing on the europa.eu. They have started accepting third party services/content, though they cannot include third party cookies (as the GDPR needs to be followed). The training will also go through the operational details (what kind of permissions each of the editors will have), following best practices, use of europa libraries, etc. Originally scheduled for march, this was not possible but this training will happen ASAP. This training will not be exclusive to EMODnet but will be horizontally done for MARE projects.

Conor Delaney (EMODnet Secretariat) indicates that we will have a development web site, where the Portal coordinators can create and edit content. Then it will go into review before being published on the production website. Carlos Cerezo (DG Mare) indicates the development web site is good for testing new functionalities. However, from an editor role perspective it would be better if they add content directly in production, but then there needs to be a publisher to review the content before it is available on the production web site. These publishers need to be aware of all the requirements posed by the commission.

For the longer term: there are rules on editorial management, copy right and intellectual property which will need to be agreed on in MARE and then made specific for EMODnet. The commission may also provide a service that they can review audio and video, which EMODnet can make use of.

Conor Delaney (EMODnet Secretariat) indicates we have already started relying on commission services, gives the example of EU Survey, and Europa Analytics used in the Central Portal. The thematic portals will have access to this dashboard, and it may start to replace the current EMODnet dashboard.

Carlos Cerezo (DG Mare) introduces his colleague, Carlos Baladron (DG Mare), he is the security responsible for DG MARE. We will discuss some security related aspects

(Extra topic) Europa Analytics

It's a Commission requirement to stop using all web analytics, e.g. Matomo, Google, especially the ones storing personal information outside of the EU space. All portals will have the same siteID in europa analytics.

EA has been upgraded to Pickwick pro, a specialised version made for the Commission. Carlos Cerezo (DG Mare) indicated that during the migration we have lost one week of statistics. This has also had further impact for EMODnet (Grafana dashbaord) as the API has changed.

We are evaluating replacing the Grafana dashboard with Europa Analytics dashboards. However, currently we only have viewer permission for Europa Analytics (EA), but we can negotiate new features with the EA teams if required (e.g. there is also a running negotiation regarding additional search statistics).

Carlos Cerezo (DG Mare) indicates that event tracking is also supported by the new EA. This gives a lot of flexibility, but we need to consider that we need to manage the personal data collected in line with the GDPR.

Moving to Europa Analtyics after centralisation will also impact the progress indicators and reporting done by the Thematic Portal. We aim to create a more automatic way to do reporting, using the EA dashboards. Some of the reporting will still need to be collected by the thematic portals (e.g. download, use of web services, publication of datasets etc.)

Feedback from lots 35mins

Was done throughout

Overview of who does what within the CP team

Joana Beja (VLIZ) indicates the different contact points for different items related to centralization (see presentations)

Metadata review and updates

Conor Delaney (EMODnet Secretariat), Francis Strobbe (EMODnet Secretariat) and Tim Collart (EMODnet Secretariat). Britt Lonneville (VLIZ)

New layers/products bugs

The information in the viewer is taken from the metadata available in the services of the portals (CSW, WMS, etc.).

At the moment, the configuration of the viewer (adding, updating layers) is a manual process, but the aim is to automate this so it can be easily synchronized.

For the GetFeatureInfo: we can create aliases for attribute names, but this needs to be informed by the portal.

In case the information of the GetFeatureInfo is more complex, the portals can do the formatting on their end using html templating, and in the central portal we can then render this html.

Removal of layers from the viewer → This needs to be communicated to the central portal team.

Old products may be retained in the catalogue, but should not all be in the viewer. This is however the decision of the portal

ACTION 4: Thematic portals are also asked to report bugs in the viewer (not missing features at this point)

Graeme Duncan (EMODnet SBH) indicates they have a number of grouped layers that show everything together, which may group multiple datasets (and metadata records).

News Service

Nathalie Van Isacker (EMODnet Secretariat) and Francis Strobbe (EMODnet Secretariat)

Central Website Content

Cécile Nys (EMODnet Secretariat) and Francis Strobbe (EMODnet Secretariat)

(added topic) Security requirements on SSL

Carlos Baladron (DG Mare) indicates the requirement the TLS 1.0 and 1.1 can no longer be used. DG DIGIT has requested to exclusively use TLS 1.2 and 1.3. since one year. However, given the new geopolitical situation, there is increased push to make sure the emodnet (commission) services are secure.

Bjarni Pjetusson (EMODnet Geology) indicates they have been in contact with Bart Vanhoorne (VLIZ) to test the Geology services. They have successfully disabled the support for TLS1.0 and 1.1, however they still get an F rating using an SSL check (Bart Vanhoorne (VLIZ) indicates it is an OpenSSL vulnerability in the library Geology is running).This may require an update on their proxy server which has a potential impact on many of their users, as such they are hesitant in making this change as they do not understand why they are still getting the F rating (the review they used is that it a man-in-the-middle attack is forced). He indicates that we are offering access to open data, so the risk is limited

Carlos Baladron (DG Mare) however indicated this may provide access to their backend, which can theoretically allow external people to make change to a service that is provided as a commission service, and this is not a risk they can take.

Carlos Baladron (DG Mare) indicates that without knowing all the details of the error flagged by Bjarni, they cannot asses, however as a best practice they would recommend to update the server. Given the reported OpenSSL vulnerabilities, they would like Geology to update the OpenSSL library on their services.

3. Helpdesk, an introduction to how the new EMODnet helpdesk works (EMODnet Secretariat) - see PPT EMODnet Helpdesk

Francis Strobbe (EMODnet Secretariat) introduces the help desk work flow, which will be centralised and using JIRA service desk.

He walks the thematic coordinators through how the system will work.

Licensed users:

• project administrators: Central Portal Staff
• Agents: Thematic portal coordinators (with a JIRA account) who will initially respond to feedback request. They can interact in the JIRA Service Desk web interface or can be interacted with through email (which will be added to the ticket)
• Collaborators: These are other people in a thematic portal group that can be brought in to respond to the feedback request. Theses have similar permission as the Agents

Unlicensed users:

• Customers: They can interact with the ticket through email, which will be logged in the ticket

Francis Strobbe (EMODnet Secretariat) shows the workflow for the feedback requests in JIRA service desk (see figure below)

Figure 1: The workflow for the Helpdesk system.

• Users can create a feedback request through the contact us page. This allows the user to send an email to secretariatemodnet [dot] ec [dot] europa [dot] eu. This email will trigger a JIRA service desk ticket to be created, and the user is notified
• The secretariat will then assess which portal should respond to the request and assign a Portal Coordinator (as an Agent)
• The Portal Coordinator (Agent) or other partner (Collaborator) can then follow up on the ticket through the webinterface or email. Internal commnets allow to communicate without alerting the user
• When the ticket is resolved, it can be closed
• The user can then rate the feedback using the customer satisfaction form

Benefits here are;

• Less email (though still possible).
• Reporting will be made easier as the feedback requests are automatically logged and can be exported.
• Customer satisfaction can also be considered as a new indicator.

Timeline for rolling out this workflow: From next month 01 May 2022 this will adopted.

Update regarding the EMODnet JIRA -

Bart Vanhoorne (VLIZ) indicated that the current VLIZ hosted JIRA is no longer possible past 2023 as Atlassian requires everyone using JIRA to move to the Atlassian Cloud

Carlos Baladron (DG Mare) mentioned that they are in a similar process for another project, where the Atlassian cloud provider is in Frankfurt, within the EU space

COMFORT BREAK – 10’

4. CP Technology Update (VLIZ & secretariat)

New Geonetwork -overview and next steps (Bart Vanhoorne (VLIZ) )

Bart Vanhoorne (VLIZ) presented on why we are updating the central portal network:

• Fully automated harvest of the metadata managed by the Thematic Portals.
• Automated, where possible synch as a source for the geoviewer config (MetaGis).
• Advantage is that GeoNetowork is a standard OGC software stack.

What are benefits of update to 3.8.3

• More complete metadata
• Automated checks on metadata content
  • Distribution URLs correct
  • Appropriate license (CC-BY)
  • Inspire check tool
• Inclusion of new tags (SDG, MSFD, EOV)
• DCAT support, allowing harvest by EU Open Data Catalogue
• Multilingual support

Version 3.8.3 used as we have a custom compiled module to allow DCAT

Final version is deployed and ready to moved to production

Next steps:

• CSW endpoints are required for all lots
• Compare DEV recor counts per lots vs. current PROD version
• Deploy in production
• Verify the completeness of records
• Decide on the metadata filters that need to be available on the central portal geonetwork interface. Input from portal needed
  • From Graeme Duncan (EMODnet SBH) to Everyone 11:36 AM (from Chat)
    I'd suggest "formats" is not necessary as a filter
• INSPIRE compliance check
  • Need to decide on an EMODnet subset of the test suites
  • Implement a UI (by Bilbomatica, 2022-2023)
• Serve as a catalogue for DTO
• Question from Dick:
  • currently many of the links are pointing to the thematic portal domains
    • these should be changed by the portals at the time of the switch to the centralsied EMODnet
• INSPIRE check
  • Will it use the INSPIRE API to validate metadata
  • We will do this in a scheduled manner (batch processing)

ACTION 5: All thematic lots to provide feedback relating to the catalogue tags/filters via the tickets linked under EM-534

The importance of following the metadata specification requests (Secretariat) - see PPT Metadata specification request

• Tim Collart (EMODnet Secretariat) gave an overview about metadata requirements specifically the distribution links
• MetaGIS content which is the background of the viewer content is done manually, not sustainable for all thematic lots, automation is envisaged in the future
• Proposal of consistency in dataset metadata records to facilitate this automated harvest/synchronisation: Service URL, Protocol, WMS layer name, File URL
• Call for standardisation for how the information is served
• Graeme Duncan (EMODnet SBH) explained on how the group layers are made available in their viewer. For EUSeaMap, the link to download each dataset is in the attribute table of the data itself. GeoNetwork has the download link to the whole dataset
• Tim Collart (EMODnet Secretariat): Would a feasible solution be to create a metadata record at group level? Graeme Duncan (EMODnet SBH), we can explain the specifics in the SBH meeting, but we can agree on what a dataset is and how we can best expose it
• Antonio Novellino (EMODnet Physics) explains that this constraint is also present in Physics. Need to think how we can expose the product without having , e.g. 12 different links.
• Tim Collart (EMODnet Secretariat): Good example is what was done for Bathymetry reference source layers, where each layer can link to the CDI entries.

ACTION 6: Secretariat and SBH (& Physics) to follow up on providing grouped or aggregated layers (e.g. wms layer groups) in the viewer, and determining how to deal with this in the metadata and the WFS services.

ERDDAP - overview of the products already migrated to it and next steps (VLIZ)

• Frederic Leclercq (VLIZ) gave a brief overview about the Central Portal ERDDAP instance, technical updates implemented
• ERDDAP was setup to facilitate the sub setting and downloading of data from the viewer. Reads a configuration file produced by metaGIS
• Overview of progress for each lot and a short summary of known issues/constraints
• Last slide focusing on forward outlook for the next few months

LUNCH BREAK – 60’ (12:30-13:30)

Tuesday 26 April 2022 13:30 EMODnet TWG reconvenes

Conor Delaney (EMODnet Secretariat) introduces the afternoon session indication Bilbomatica ( Natalia Orio and Clara Becares) will present the map viewer, which will be the central access to EMODnet data, being API agnostic, providing access to ERDDAP and WFS.

5. Map Viewer progress update (VLIZ)

Clara Becares (Bilbomatica) presents the new central map viewer on the development server: https://emodnet.development.ec.europa.eu/geoviewer-new/#

She shows the different tools available:

Download and sub setting tool: to download multiple EMODnet products

Figure 2: Downloading from the new map viewer

It also shows the option for the vessel density layer to select the time dimension

From Graeme Duncan (EMODnet SBH) to Everyone 01:46 PM (chat)
Are there plans to make the download tool cross-service applicable? For example it will use WFS for vector layers that have WFS enabled, and also ERDDAP all through the same interface for the end-user?

Currently not yet implemented by the download and sub setting tool will work for WFS services for vector type layers.

Filtering tool: layers can be filtered according to the fields available in the config file (need to be informed by the portals).

Time animation tool: time animation can be made for layers with a time dimension.

GetFeatureInfo tool:

Information can be displayed in multiple ways:

• if just JSON format, the information is displayed as a table;
• if in HTML format, the information is rendered as HTML in the way the portal has formatted it using the GeoServer GetFeatureInfo template functionality.

Marine Regions tools: Allows to display the marine regions boundaries on the map.

Embed tool and share tool: Users can share and embed a link with the exact configuration of the users map.

Active layer: Some layers have additional functionalities which can be reached by activating the layer.

Review of what layers are to go into the Map Viewer

Joana Beja (VLIZ) indicates that it is important for the portals to follow up in the JIRA to the questions posed by the Central Portal team, as this will allow to configure the viewer with each of the map layers of the thematic portals.

In some cases, the number of layers in the CP map viewer may be condensed (though showing the same information) compared to the thematic portals

Next steps - timeline out the launch

The aim is to have the new map viewer released by 01 Sep 2022.

This release should include the level 1,2 &3 priority functionalities.

Will this occur simultaneously with the centralization.

6. Overview of progress for each lot (VLIZ and Secretariat)

Conor Delaney (EMODnet Secretariat) shows the priority list of the Central Portal Viewer.

These functionalities will be there during the release in September. However, the portals need to follow up with the CP team so that the necessary layers are there.

Conor Delaney (EMODnet Secretariat) and Joana Beja (VLIZ) give a short updated on the progress with each thematic portals.

Main message: the technical problems have been solved but we need to populate the configuration files so that all the layers of the portals are displayed.

Chemistry still requires some work to display specific seasonal data

SHORT COMFORT BREAK - 15’

7. Updates and progress since previous Technical Working Group Meeting (Tim Collart, Conor Delaney, Joana Beja, Francis Strobbe)

Action List from 10th TWG (Update this and point out that it occurred later)

The EMODnet Secretariat ran through the previous (10^th) TWG meeting Action List. There were no further comments on the actions.

Question of Bart Vanhoorne (VLIZ): should the TWG and SC be hosted as an event on the EMODnet Central Portal events calendar: Indeed this can be advertised on the CP.

ACTION 7: Francis Strobbe (EMODnet Secretariat) to create a calendar event in the Central Portal to communicate this meeting to the EMODnet consortium

Jira: Overview of usage (EMODnet Secretariat) - see PPT JIRA update

• Francis Strobbe (EMODnet Secretariat) provides an overview of the JIRA activity, with a lot of tickets being created and resolved. This shows the clear adoption of JIRA as a central tool to achieve the centralisation. All portals are involved in using JIRA in fairly equal proportions. Epics are now used to group categories of tickets to specific issues.

OGC service monitoring Update & INSPIRE compliance and updates - see PPT INSPIRE compliance and updates

• Tim Collart (EMODnet Secretariat) gave an update of the metadata and data URL's in Service GetCapabilities:
  • Metadata URLs are there but cannot be parsed;
  • Linkage from metadata to service;
  • Monitoring quality of service (INSPIRE) has improved well in the last months. Most services are now meeting the INSPIRE requirements;
  • Implementing new INSPIRE checks (2 level): Quality of service metrics & INSPIRE validator to (metadata content). INSPIRE validator will be implement with CP and Bilbo;
  • SBH service, one is quiet low, altough running on the same geoserver.. Response time is higher. SBH to look at the graph on monitor.emodnet.eu. Tim Collart (EMODnet Secretariat) to look into if it's first bit or last bit. (Done during meeting);
  • Metadata parsing. What about external records ? SBH to harvest this records and CP will then harvest SBH.

Review and discussion on the collection of metrics for reporting (EMODnet Secretariat/CINEA)

Conor Delaney (EMODnet Secretariat) presents the metrics we are collecting.

• Indicators removed:
• Visual harmonisation score: No longer any point to collect this as we are now on the Europa Domain and we are restricted to the rules set by the commission.

Conor Delaney (EMODnet Secretariat) note that not everyone used the latest template in this quarter.

ACTION 8: Portals to use the latest templates of indicators. Secretariat will reject quarterly reports in case they do not follow the templates

Conor Delaney (EMODnet Secretariat) mentions that long term goal is to collect indicators in a more automatic way. Here the migration to Europa Analytics is important. Also data not in EA should be collected automatic. This includes:

• Coverage Indicators: in the contract, the EEA MSFD sea basin shapefiles should be used for the coverage recording. While this is not a requirement , we will need to go towards this in the future. The Secretariat has a deliverable on this that will report on these coverage indicators.

Conor Delaney (EMODnet Secretariat) emphasises that we will need to make sure all the licenses are updated to CC BY 4.0.

Conor Delaney (EMODnet Secretariat) states that the use case statistics were not available for Q1 2022, due to migration of Europa Analytics to Piwik PRO which changed the API.

1. Overflow from previous session - space for discussion

Questions of Graeme:

• On coverage indicators, many areas not covered by EEA shapefile (e.g. Arctic)
• Will there be a push for the new OGC API standards in EMODnet (https://ogcapi.ogc.org/):
  • At the moment we will keep using the original standards, as this is what is currently documented by INSPIRE (form back in 2013);
  • However, Inspire is updating the technical guidance documents, though this has not been done for services ( https://github.com/INSPIRE-MIF/technical-guidelines/tree/2022.1/services);
  • Bart Vanhoorne (VLIZ) indicates there should however be a push to move to the STAC standard in light of the DTO.

Comments of Carlos:

• Data protection in terms of Europa Analytics, all data collected is anonymized, but first the information is extracted (e.g. location of IP). In terms of aggregation, the daily data is aggregated 3 times a day, daily data aggregated weekly and . Raw data is automatically removed after 30 days;
• Replacement of newsroom to mailchimp: One downside of this is there is no way to track mailing lists;
• Requirements to provide EA access to different thematic coordinators. Conor Delaney (EMODnet Secretariat) indicates everyone should have access but will check again.

ACTION 9: Conor Delaney (EMODnet Secretariat) to check if all the coordinators that need EA access are there and if not instuct them on the necessary steps.

2. Wrap up, actions, next meeting*.

Next Technical Working Group Meeting: format, date and location (Secretariat & all).

Request of Zoi Konstantinou (EC, DG MARE) : At least one physical TWG and SC a year. The next meeting should thus be a physical meeting in October-November 2022

End of Meeting

Wednesday 27th April 2022

09:30: EMODnet SC Joint Session*

*This Session was open for Members of the EMODnet Technical Working Group (E-TWG) and the EC Marine Knowledge Expert Group (MKEG).

The minutes of the joint session are contained in the separate minutes of the 16th EMODnet Steering Committee.

Annex I: List of Participants

[1] The numbers for the section headings correspond to the sections in the agenda for the meeting.