13th EMODnet Technical Working Group Meeting

Spring 2023

Hybrid meeting: Physical meeting at BRUSSELS & online connection

Dates: Tuesday 4th April 2022 (13:30 - 17:30) DG AGRI, Wednesday 5th April 2023 (10:00 - 13:30) DG Mare. All times in Central European Summer Time, CEST.

Meeting Chair: Zoi Konstantinou (EC, DG MARE), Conor Delaney (Technical Coordinator, EMODnet Secretariat).

List of Participants: See Annex I

Actions list – To be tracked in JIRA.

Meeting Minutes

Tuesday 4th April 2023 – 13:30 - 17:30 (DG AGRI)

13:30 EMODnet TWG convenes.

1. Welcome & introduction (Zoi Konstantinou (DG MARE) & Conor Delaney (EMODnet))

• Welcome, recap of guidelines for remote meetings and agenda for the day.

2. Updates and progress since previous TWG (Secretariat, VLIZ)
   1. Action list from 12^th TWG

Conor Delaney (EMODnet Secretariat) ran through the previous (12^th) TWG meeting Action List, see Table 1 below. A summary of actions is listed here:

• Action 1-15: Done
• Action item 15: Thematic portals to follow up on checklist provided in EM-604. Not all portals used this. Some feedback came from SBH which was useful. Bathymetry created their own checklist.
• Action 15: Tools and Guidelines section. TWG asked to review the section.
• Action 19: A bug that will be updated in the next release of the viewer. The MapViewer was using the tabledap formats, not the griddap formats, relevant for EMODnet Bathymetry DTM.
• Action 27: Bart Vanhoorne (VLIZ) indicates that this is not immediately feasible since the viewer is dynamic.
• Action 34: Controlled vocabulary as proposed by Dan Lean (EMODnet Biology). Has not been progressed but remains a good idea we should follow up on.

Table 1: Action list from the 12^th TWG.

1. 2. JIRA overview and usage (Tim Collart, EMODnet Secretariat)

Bart Vanhoorne (VLIZ) Led a discussion on funding of JIRA/Confluence. Currently VLIZ has covered the cost of JIRA, but this needs a better solution. Moving to the cloud will be necessary for the new version. This will be a cost of 15K.

ACTION 1: JIRA Service desk stores personal data, need to select a cloud in the EU space, for GDPR compliance when JIRA moves to the Cloud

Tim Collart (EMODnet Secretariat) presented a review of the status of the JIRA tickets

Status of JIRA tickets

• 744 tickets created, 75% completed.
• More tickets closed than created.
• Tickets created by lots should be closed by thematic lots once they're happy all actions have been done.
• Tickets assigned to thematic lots should be set to review once the lot has done everything so that the CP team or Secretariat can close it or reassign if something is still missing.

ACTION 2: Conor Delaney (EMODnet Secretariat) to review the number of users for JIRA.

ACTION 3: Bart Vanhoorne (VLIZ) and Conor Delaney (EMODnet Secretariat) to review modules in place and whether some can be removed/disabled.

1. 3. OGC service monitoring Update and INSPIRE compliance and updates (Tim Collart, EMODnet Secretariat)

OGC service monitoring

• Progress on this task can be followed on EM-14(CINEA - Action on Web Services MetadataUrl and DataUrl fields (21/11/2019) [Previously JIRA ISSUE 33])
• Bart Vanhoorne (VLIZ) stated that when Geonetwork is updated the INSPIRE extension will be installed. The INSPIRE extension was developed by Bilbomatica.
• Tim Collart (EMODnet Secretariat) gave an overview of getCapabilities for all portals (see presentation, EMODnet_13thTWG_April2023_OGC_service_monitoring).
• Monitoring webservices: Bart Vanhoorne (VLIZ) stated that GeoHealthCheck, updated every 10 mins.

3. Technical Updates from EMODnet Thematics & Data Ingestion

In a change to the agenda the TWG agreed to allow EMODnet Biology to bring forward their presentation on R-Universe as due to availability of speaker.

1. 1. R-universe for EMODnet GitHub (Maëlle Salmon EMODnet Biology))

Maëlle Salmon (EMODnet Biology) presented the R-Universe they want to create for EMODnet

• R-universe is an rOpenSci Project, relevant for R packages and analysis projects. There are a few R Packages for EMODnet. Infrastructure to distribute packages is built in a git repository.
• Easy for users to install packages from CRAN (Comprehensive R Archive Network). But not all packages are relevant for CRAN
• R-universe builds binary packages which is easier for MS Windows users.
• Maëlle gives a few examples of R-universes for different research communities. It would be good to have an EMODnet R-universe listed there.
• What task are needed to do this?
  • Create a GIT- Repo named R-universe.dev on a GitHub account contained the packages JSON file.
  • We need to keep the documentation up to date (important for SEO)
• We could do 1 EMODnet R-universe, rather than a separate one for each thematic.
• Biology will take the lead on the implementation.
• What are the requirements from the thematic lots?
  • The package maintainers would have to keep the packages, or shiny apps up to date.
  • You can choose which packages go into the R-Universes (e.g., you can keep old packages or packages under development out).
  • Biology R-Packages published in the EMODnet repo are ready to do, they are no longer under development.
• Not all portals use R, some use Python instead. Python maybe a next step to create a user community. R-Universe is a good start.

ACTION 4: Biology to contact each of the lots, to gather which portals want to publish an R Package in the Universe.

1. 2. EMODnet Bathymetry Updates

Updates EMODnet Bathymetry

Dick Schaap (EMODnet Bathymetry) presents.

New product (2022) will be released in April.

• It is based on a bigger set of data.
• It also covers data in the Caribbean.
• It includes a new dataset of coastlines, based on Sentinel 2 & Landsat.
• Can be downloaded as tiles, or as a subset in the ERDDAP.
• Will also be offered through the WMTS.

ACTION 5: Secretariat to add the WMTS details to the documentation on github.

• The number of High res DTM's have been increased with 19 new ones. Some of them are very high resolution. They are integrated in OGC services and in the EMODnet Map Viewer and are downloadable through it.
• The number of datasets has been increased. Now more than 20 data partners (Hydrographic Offices). From 30k to 41k. Composite DTM have been increased from 270-274.
• User stats: More than 100k downloads, ca 12k users, 68 Mil OGC service requests

Migration actions:

• Bathymetry narrative was adapted to fit the central portal.
• The technical reports have been transferred to the central portal.
• Every layer is now documented and downloadable in the catalogue and can be accessed through the map viewer.
• New metadata Records added to Sextant catalogue and new CSW services created to have a single point to harvest by the Central GeoNetwork.
• They added a new WMTS for the 2022 DTM.
• They produced a single merged NetCDF file (previously done by CP TEAM) which is used by ERDDAP.
• They added an additional viewer layer with the DTM tiles, from where users can download tiles in different formats. Useful for many users.

Remaining wishes. The central portal is mature but there are things that we can improve (as requested by users):

• Some formats are not fit for Bathymetry (GeoTIFF & ESRI ASCII are the most important)
• Some issues with loading the ERDDAP netCDF in QGIS.
• Improve get feature dialogue for source reference layer and survey data sets layer.
• Improving depth information from map click.
• Bathymetry request more information:
  • About the new KPIs.
  • Information about downloads from central portal.
  • Issue with the survey track layer flagged by Sissy Iona (EMODnet Data Ingestion)

ACTION 6: Bathymetry to add specific technical issues and proposed improvements to JIRA

1. 3. EMODnet Biology Updates

Joana Beja (EMODnet Biology) presents.

• There will be a contract gap between the current phase IV and the next phase.
• They are currently doing the last data harvest of phase IV. Will include existing published datasets as well as new datasets:
  • Mostly from Consortium.
  • Ad-hoc submission from EMODnet Ingestion and other organizations.

New Data Products:

• Two products.
• R-Package that facilitates working with the species density data.
• Foreseen to publish 4 more.

New Use Cases:

• Published a new one.

Other deliverables:

• Community workshop outcome will be delivered soon. This was led by HCMR.
• They are working on a peer reviewed paper on EMODnet Biology.
• Launch of the European MBON node:
  • Landscape has changes, and MBON node already exists.
  • Instead, Biology will align with it.

Technical actions:

• Publication of the products is not straight forward:
  • NetCDF files for Biology are not easily ingested in ERDDAP (even though they meet Climate and Forecast (CF) metadata conventions).
  • ERDDAP and GeoServer don't have a seamless implementation for biological products.
• Biology sees that there is a need for new functionalities in the viewer.

Updates:

• Aim to have all EMODnet Biology metadata records in the EMODnet Catalogue, by end of phase VI.

Cross-Thematic Interactions:

• Strengthening collaboration with Seabed Habitats (SBH):
  • Biology will integrate data submissions from SBH partners.
  • Biology will follow SBH guidance on EUNIS-VME collection so that it can be implemented in Biology Databases.
• For phase V Biology will collaborate with Geology.

External interactions:

• Participated in various events.
• Presented at IODC-II.
• Involved in various projects and proposals so that the datasets can flow to EMODnet Biology and other EMODnet Portals.

Outlook:

• Specific requirements for Phase V that will need to be addressed:
  • Genomic data publications.
  • Definition of KPI's to show progress between thematics and RSC/ICES/other EU funded projections.
• Continue to explore opportunities with other OBIS nodes. Criteria is the quality level required by EMODnet Biology.
• Specific actions for data training and publication from Southern Mediterranean Countries.

Discussion between TWG and Biology:

Question from Graeme Duncan (EMODnet SBH): He would like to have a discussion with Biology & Dan Lear (EMODnet Biology) regarding a use case.

Question from Dick Schaap (EMDOnet Chemistry): Define KPI's with RSC & ICES. EMODnet was asked to define the KPI at this stage (e.g., use cases). This is a tender requirement, not something that the lots going for Phase V decided to implement by themselves

1. 4. EMODnet Chemistry Updates

Updates EMODnet Chemistry

Dick Schaap (EMDOnet Chemistry) presents.

Good progress:

• Data gathering is continuous. Micro litter is a new development, still working to define standards.
• Using harvested data collections for new aggregated and validated data collections.
• Preparing derived gridded DIVA maps for eutrophication, so these can be visualized as time series.

Current contract till October 2023

Chemistry supports MSFD implementation:

• Chemistry established as the reference for litter data in MSFD.
• Comparable role for contaminants will be assigned to Chemistry (as communicated by JRC).
• This will result in more partners providing MSFD monitoring data into EMODnet.
• There are requests for extending to other chemical parameters.
• Several Use Cases from RSC (Regional Sea Conventions) which combines chemistry data with data from other services.

Centralization actions:

• Moved static content to central portal.
• Transferred reports and guidelines to central portal.
• They prepared a lot of metadata in Sextant. Harvesting is going well. Updates made by Chemistry in Sextant have been applied efficiently.
• Continuation of OGC services for map layers of Marine Litter: Contaminants, CDI Records per P36 and Locations.
• Kept the ODV portal active to provide additional functionalities not in the CP viewer.

Remaining wishes:

• Download formats from ERDDAP to be reviewed.
• Improve Get Feature dialogue for CDI data sets layer.
• Improve colour ranges and legend for DIVA gridded maps. They want users to select the ranges.
• Direct Links from product records to CP Map Viewer folders.

Discussion between TWG and Chemistry:

Grigore Rischitor (DG MARE): EEA and JRC indicate that they appreciate the work done by EMODnet Chemistry, and in the future, they will push for additional indicators to be hosted by EMODnet Chemistry.

Dick Schaap (EMODnet Chemistry): They are happy with this, but it will come with additional requirements, and this needs to be considered with the funding.

1. 5. EMODnet Geology Update

Updates EMODnet Geology

Uffe Larsen (EMODnet Geology) presents.

Uffe Larsen (EMODnet Geology) gives an overview of the EMODnet Geology architecture (GeoServer/PostGIS/EGDI metadata/zipped products on a web server).

Users can interact in two ways with the Geology products:

• Use the map viewer or OGC services to extract the data.
• Download the zipped files from their servers.

Uffe Larsen (EMODnet Geology) indicates the changes to the Geology Layers in Phase 5 (will be deployed on GeoServer in August 2023). The central portal technical team will try to deploy the changes as soon as possible in the viewer when they are available.

Discussion between TWG and Geology:

• They used to have a form to collect user input. This is no longer allowed (according the GDPR), but they still need to report on this in the quarterlies.
  • Conor Delaney (EMODnet Secretariat) indicates that this will be discussed tomorrow.
• For the next phase, they are discussing to publish geotechnical data. This data comes as ASCII files. Where can these be hosted?
  • Central portal team: If it needs to be visualised, they will need to host it on GeoServer.
  • Uffe Larsen (EMODnet Geology): from each file (location on map), they would like to have an ASCII document.
  • Dick Schaap (EMODnet Data Ingestion): it could be hosted in SeaDataNet. This would be the best approach from a data management perspective.
• Question from Uffe Larsen (EMODnet Geology). The description of dataset attributes is often just a word document, is there a standard for this?
  • From CP Team. Yes, can be configured as "ISO FeatureType Catalogue" Some documentation here: https://docs.geoserver.org/latest/en/user/extensions/csw-iso/mapping.html.
• Various suggestion from TWG colleagues on where to store layer zip files, SBH approach is via https://www.ags.org.uk/data-format/, Dick Schaap (EMODnet Bathymetry) suggested Sea Data Net, Human Activities serves their files from their own GeoServer instance.

Carlos Cerezo (DG Mare) indicates that in case there is a need for user data forms to be integrated, this can be done but there is a procedure.

Conor Delaney (EMODnet Secretariat) reminds colleagues that at a previous TWG meeting, CINEA waved the requirements for this form data to be collected. It will be further explained tomorrow.

1. 6. EMODnet Human Activities Updates

Nick Earwaker (EMODnet Human Activities) presents.

Centralisation Update:

• Set up redirects & turned off old portal.
• Moving data downloads from old portal web server to the GeoNetwork data catalogue.
• They started recording download numbers from GeoNetwork instead.
• Delivered Vessel/Route density as a time series (using ImageMosaic in GeoServer) and removed old vessel density layers used in the old portal.
• Checking the metadata and data links in the web services
• They show a new diagram of their architecture, now much simpler.

Ongoing work:

• Deliver the Route/Vessel Density layers as netCDF files. This will allow them to be used in ERDDAP and by the download and subset tool.
• They are looking into better organising the data presented in GetFeatureInfo using HTML templating for some layers (e.g., Fish catches)
• They show a list of all the updated datasets.
• They are producing the vessel density data for 2022.
• Layer update process. Fairly seamless, though there is now an extra step.

1. 7. EMODnet Physics Updates

Marco Alba (EMODnet Physics) presents.

Centralization update:

Continuously updating the services and products and fixing problems as soon as they appear.

• Data:
  • Reorganisation of data collections under the filters for Physics (needed to simply access to users).
  • Update the HTML pages presenting the in-situ data.
  • Clean the service offering the in-situ metadata (bug later reported).
• Products:
  • Issue with animation tool (the fix will be part of the next release of the map viewer).
  • Advanced features product pages to be linked from the static page content (Physics are working on it).

Deliverables:

• They are on plan for all the deliverables, but some delay:
  • • Integrating high resolution in major rivers mouth data (WP2).

They have started working on their products for inclusion in the central portal viewer.

They have received feedback from their users on the new centralized portal.

• Generally positive, handy that everything is available from a single viewer.
• Some missing features from the old physics portal.
• Users get lost in "Layers" and "Catalogues".
• Issues to be fixed with the viewer: GetFeatureInfo pop up sometimes over the filter window and window sizing is not optimal.

ACTION 7: Physics to create JIRA tickers on these bugs.

Meetings/events:

• Participated in 28 meetings/events.
• Contributed to organizing the Ocean Race in Genoa.

They are working on improving performances (testing hosting using Amazon Web Services):

• Hosting Total Suspended Matter (TSM) layer has been problematic.
• They are now testing a cloud deployment, using the scalability features.
• First results show a better performing WMS layer.

EMODnet Physics raised the point that the using the EMODnet portal can be difficult for users.

The EMODnet Secretariat is working on making it more intuitive for users:

• An FAQ is on the way.
• Marco Alba (EMODnet Physics): notes that help needs to be easily accessible to users. Idea of an interactive help module could be an interesting long-term development for the viewer.
  
  8. EMODnet Seabed Habitats Update

Graeme Duncan (EMODnet Seabed Habitats) presents.

Centralisation update:

• They did the URL redirects.
• They migrated the static download files.
• Metadata records have now been programmatically updated:
  • To remove the links from the old thematic viewer.
  • To update the links to GeoServer.
• Updated guidance documents on accessing web services, habitat map submission process.
• Help desk process testing:
  • It is working fine but they have some issues when people are out of office to reply to the ticket in a timely manner.
    • Conor Delaney (EMODnet Secretariat) replied: Physics have implemented a helpdesk group email. SBH already have a helpdesk so this can be implemented.
    • They had replied directly to email (not using service desk). Some users may need some training.

Centralisation updates still to do:

• SBH reports still need to be added.
• A large number (low priority layers) still need to go onto the portal. Work to condense the number of layers.
• Request to add a free text advanced filter in some feature release.
• They still need to update their data products summaries.
• Updating the CRS:
  • CP_Team commented that they are working to update the viewer, so it is in web Mercator, so SBH was asked not the make changes to the CRS at this point.
• They had a few users asking for training and help documents for the viewer: This was addressed by Conor Delaney (EMDOnet Secretariat) in previous section.
• They are waiting on more formal procedure for data product updates.

Open Sea Lab:

• Helen Woods and Eleanor Stewart represented SBH.
• Trailed some new raster EU Sea Map layer, this could increase the performance of the layer (now complex vector layer). They are waiting on feedback of users to see if this raster layer is useful or not.

New datasets:

• 80 new datasets will be added.
• They have an issue with cascaded WFS services, which crashes their services when it goes down. They will create a data cache to work around this.

Other updates

• Working towards OBIS integration for point data submission.
• New EUSeaMap is underway.
  • Other composite products will be updated at the same time.

1. 9. EMODnet Data Ingestion Updates

Updates EMODnet Data Ingestion

Dick Schaap (EMDOnet Data Ingestion) presents.

Good progress.

• There has been an increase in submissions.
• 180 data originators from across sectors (many NGOs) submitting data on litter.

There are two stages of datasets processing levels:

• Data is published as is with metadata (698 datasets)
• Data is elaborated to become fit for the European Data Management systems (e.g., EurOBIS, SDN, etc.) (566 datasets)

New developments:

• Working with the Crown Estate Marine Data Exchange through MEDIN. All data on the UK shelf needs to be added. A selection of useful datasets has been made. They are connecting an exchange between MEDIN and Ingestion.
• Upgraded and expanded Sensor Web Enablement demonstrator for data streams with (near) real time data into EMODnet Physics.
• Improved and documented availability of data as part of offshore licenses for renewable energy, aquaculture, and other Blue Economy activities. The data required for this licensing (Environmental Impact Assessments) usually doesn't end up in data centres. Now they have created an inventory of the stakeholders that provide licenses. Next steps:
  • they will work with them to understand their data management procedures.
  • They will then organize workshops on data management best practices and discuss opportunities for making this data re-usable (through EMODnet).
• Migration of Ingestion to the Central Portal, kick off meeting planned on 07 Apr 2023

Discussion between TWG and Data Ingestion:

Grigore Rischitor (DG MARE): Are license stakeholders contacting Ingestion, or is ingestion actively reaching out to them?

Dick Schaap (EMODnet Data Ingestion): Clarifies the process as documented above. For now, they are trying to build the landscape of contacts as well as data management procedures. The next steps will be to get in contact with the most promising stakeholders. It would be useful to have specific use cases. Some sectors (e.g., offshore wind) are already actively publishing their datasets in EMODnet, but for other businesses, the landscape is completely different. This will also be discussed at the SC, where EMODnet for Business is trying to expand in the tourism sector.

1. 10. EDITO Update

Frederic Leclercq (VLIZ) presents.

Update on project, part of the Horizon Europe Mission.

Main aim is providing the infrastructure backbone for the DTO by combining EMODnet & Copernicus data in a data lake.

Frederic Leclercq (VLIZ) shows an architecture diagram with all the components:

• Interfaces will need to have easy access to data lake, efficient but also findable data.
• Sister project EDITO Model Lab which will be creating models that will need to plugin to the EDITO infrastructure.
• They will build data pipelines that are cloud optimized to move EMODnet data into a data lake, from where end users will be able to subset the data. They are looking into using the ZARR file format and COG (cloud optimized GeoTiffs) and use a Spatial Temporal Assets Catalogue (STAC) to catalogue them.
• End user will search for parameters, not datasets, STAC catalogue will be used to allow for data to be found.

April 7th there will be a website with more info available.

Discussion between TWG and EDITO:

Alessandra Giorgetti (EMODnet Chemistry) asks what the effort required from the Thematics will be.

Bart Vanhoorne (VLIZ) indicates they will need input from lots on data formats and data model remapping questions (and there will be budget for this). But for the moment they cannot yet provide more detail (likely in the next few months).

Alessandra Giorgetti (EMODnet Chemistry) indicates that this will be planned in a timely fashion. What is the timing of this project?

Bart Vanhoorne (VLIZ): It started in October 2022, and will run for 2 years.

17:00 Session Closes.

Wednesday 5th April 2023 10:00-13.30 (DG Mare)

4. Updates from Central Portal technical team on Centralization progress.
   1. Bug lists, what problems remain?

Bart Vanhoorne (VLIZ) presents.

• Bart Vanhoorne (VLIZ) reviews the remaining bugs and snags of the viewer. 15 issues remain to be solved.
• Questions from the coordinators:
  • Bart Vanhoorne gives additional info on the GeoFabrik data layer issue: It's a customized Open Street Map (OSM), this was provided by Bathymetry, but VLIZ will take over the administration. Dick Schaap (EMODnet Bathymetry) indicates that this is useful as OSM sometimes put things out in the ocean which can overlap with EMODnet data. Bart Vanhoorne indicates there is now also an "OpenSeaMap" which we can consider using in the future, but for now it is still too early in its development.
  • The GeoNetwork version used was produced by the Dutch government, which had a few Dutch license terms hardcoded in the DCAT which have now been remove.
• Chemistry responds to list:
  • THREDDS data layers names: 80% fixed. Spaces have been removed. DIN layers still have brackets "("")".
• Physics responds to list:
  • Pop up of Getfeatureinfo covers the filter window.
  • Too many requests sent by central portal animation is blocked by Physics server. We will till new release of the Map Viewer where the animation sliders have been fixed.
• SBH responds to list:
  • Could layers be filtered with free text operator? To be discussed between SBH & CP team.
  • Question from Uffe Larsen (EMODnet Geology): They also have a date they would like to filter on with a free text field. To be discussed with CP

ACTION 8: Portals to check the issues listed and verify if any of these could also be applied to their data layers.

ACTION 9: If there are additional improvements that need to happen, to create a JIRA ticket describing the issue.

1. 2. Future developments post launch
      1. EMODnet Ingestion centralisation

Joana Beja (VLIZ) informs participants that Kick off meeting planned on 07 Apr 2023.

1. 1. 2. Move to Drupal 9/10 & Move to ECAS login.

Bart Vanhoorne (VLIZ) shows the timeline of three major upcoming developments on the Central Portal:

• Migration to Drupal 9 or 10 to take place May - August 2023.Drupal 9 or 10 will depend on the availability of the ECAS component (if Drupal 10 component is ready, we will move to that immediately, as this would be ideal)
• ECAS integration expected for Jul-Aug 2023. User authentication on europa.eu is required to use the ECAS authentication system. Initially we will implement for the access to the DRUPAL CMS.
• Ingestion centralisation planned from Apr to end 2023. They may also have the requirement to use ECAS to authenticate the submission process.

Question from Dick Schaap (EMODnet Ingestion/Chemistry/Bathymetry): Do you need support from the Commission to use ECAS?

Bart Vanhoorne (VLIZ) answered that there is a library that can be used.

ACTION 10: Bart Vanhoorne (VLIZ) to send link of ECAS Drupal module to Dick Schaap (EMODnet Data Ingestion).

Question from Dick Schaap (EMODnet Data Ingestion): Can ECAS be integrated in any federated system?

Carlos Cerezo (DG Mare) answered that it can be done, but that he hasn't seen it being implemented.

1. 1. 3. Meta review process

Metadata review

Tim Collart (EMODnet Secretariat) presents.

Since centralisation all metadata is available via one Geonetwork instance. We can now have other catalogues (international/European) harvesting our catalogue. GEOSS, EU Data platform (we needed the DCAT support), INSPIRE portal and Ocean InfoHub.

Tim Collart discussed the best approach to harmonise metadata and presented various items where harmonisation can be achieved, by providing examples of implementation done right and wrong. We don't want to change the information (e.g., license information) but it's how it is reflected in the metadata, so that in the EMODnet catalogue it always looks the same way and the information can always be found in the same place across the thematic products.

Tim Collart requested that SBH share their experience in scripting automated metadata record publication.

ACTION 11: Secretariat to circulate a document with a proposal for how to best implement the metadata fields (distribution and constraints)

1. 1. 4. Update on Metadata duplication detection and INSPIRE validator.

Bart Vanhoorne (VLIZ) presents.

EMODPACE Metadata duplication detection

• This tool will detect duplicates in our metadata catalogue. It was an EMOD PACE deliverable but will be used for the EMODnet catalogue. Does fuzzy matches, checks pre-defined fields. Tool was delivered in beta mode, still needs some work and hopefully will be ready by mid-2023.
• Bart Vanhoorne showed an example that might be an exact duplicate in the SBH. Two records with the exact same title.
• Report from the checks will be made available to all the Thematics so that it can be checked, and updates done where needed.

INSPIRE Validator

• Tool developed by Bilbomatica and VLIZ that call the INSPIRE validator API and presents the output in a human readable file.
• There will need to be a selection of the fields relevant to EMODnet
• Currently in alpha version, so still needs some work.

Carlos Cerezo (DG Mare) Asked how often the checks will be run. Is it to be defined? Could it be daily, weekly, or monthly. What are the operations?

• Bart Vanhoorne (VLIZ) answered that VLIZ will be running it, just like the OGC HealthCheck Tool. The compliancy is the responsibility of the lots.
• Conor Delaney (EMODnet Secretariat) added that it should fit within the existing reporting framework (6 monthly progress reporting or quarterly monitoring).

Graeme Duncan (EMODnet SBH) indicated that GeoNetwork also has an "INSPIRE validator" tool, will we use that? The tool that Bilbomatica developed does not integrate with GeoNetwork. Bart Vanhoorne is aware of the INSPIRE check in GeoNetwork, but we have not investigated using it. It is also a question if it meets our needs, as we only want to do a subset of the INSPIRE checks.

1. 1. 5. Time stamps (Several lots include timestamps in their WMS services, this is usually presented in the format YYYY-MM-DDTHH:mm:SS.sss but sometimes it's more useful to show only part of the date, e.g. YYYY or YYYY-MM)

Joana Beja (VLIZ) presents.

Joana Beja (VLIZ) wants to trigger a discussion that could be useful for several of the Thematics. MetaGIS reads the timestamp from WMS. As such the timestamp does not need to include the full timestamp if it only a data or a season.

For NetCDF in GeoServer, you can only do full dates as it is read from the data.

Question from TWG: Can MetaGIS change the display labels of the timestamps as they are in the WMS?

• Frederic Leclercq (VLIZ) answers that you can edit the labels, but it is not fully implemented for the animation.

ACTION 12: Lots to review their time enabled layer in the viewer and communicate to the Central Portal team on how the labels need to be changed.

5. Discussion on protocols for updating the CP post launch
   1. Updating the Reports Section of the Central Portal

Protocol documents

Joana Beja (VLIZ) presents.

• Access to EMODnet CP Drupal CMS:
  • There can only be one user per Thematic. If multiple people need access, this can be a shared email (e.g., helpdesk email)
    • Carlos Cerezo (DG Mare) comments that once we move to ECAS, the accounts cannot be at organization level. Out of security considerations, an ECAS account should be personal to 1 individual.
    • Login to https://emodnet.ec.europa.eu/user

ACTION 13: Thematics to send the email they want to have access to the CP Drupal CMS

• Permissions: they cannot easily set permission for specific pages. For now, access is restricted only to all static content pages for now.
• Joana Beja (VLIZ) next presents the workflow for publishing documents in the DRUPAL CMS in slides (to be shared with the Thematics).

ACTION 14: VLIZ will also make a written guidance document on how to publish reports.

• Issues on Drupal:
  • Some of the DRUPAL CMS forms are a bit visually confusing because of some conflicts with the EC webtools (VLIZ is investigating it and will try to FIX it)

• Zoi Konstantinou (DG Mare): CP team does not have the resources to do all the publishing work. This responsibility lies with the portals.
• Dick Schaap (EMODnet Ingestion/Chemistry/Bathymetry) indicates that the thematic want to do this themselves as much as possible. He also asked if the tools and guidelines will be manageable by the Portals.
• Nathalie Tonne (EMODnet Secretariat): For now, the Portals will be able to upload the reports, but the updating of other web sections (e.g., tools and guidelines, thematic narrative) will be a step in the future.
• Alessandra Giorgetti (EMODnet Chemistry): Asked whether the thematic lots will eventually be responsible for other updates, e.g., news, static content, ...
  • Conor Delaney (EMODnet Secretariat) replied that is the aim, but we're busy with Drupal updates and ECAS integration before all of this can become definitive.
• Alessandra Giorgetti (EMODnet Chemistry): What will be the final responsibilities of the lots:
  • Conor Delaney (EMODnet Secretariat): Eventually, the lots will be responsibility for their own sections of the websites (thematic narrative, tools and guidelines, reports). After the new Drupal is live, we will also try to set the restrictions so that each thematic only has access to their own section. News will still have to be published by the Secretariat (the Thematics must use the template to prepare the content) but a longer-term goal will also shift this to the lots.

1. 2. Adding New Products

Joana Beja (VLIZ) presents.

Instructions to publish new products are documented in EM-625 - Phase IV products.

Thematics should create a separate JIRA ticket per product they want to publish. It should include all the information fields listed in EM-625 - Phase IV products.

• The layer name and subtheme it must go in.
• Product file and/or web service information.
• A metadata record, we need metadata record UUID.
• Attributes to be displayed in the GetFeaturInfo box.
  • If it needs to use HTML returned by the GeoServer.
  • In other cases, portal need to be informed what attributes need to be shown.
• Filters:
  • Which attributes, what values, animation?
• Important:
  • If you make changes in your web services (e.g., changed layer names), this will break the viewer (MetaGIS is not updated automatically).
  • CP_team aims to make the updates in MetaGIS within 2 days after being made aware in JIRA.
  • If you need new functionalities, let us know in a JIRA ticket.
  • There is a staging environment where you can test new layers. Portals need to test that it works as intended and then greenlight it for pushing into production by the CP team.

Question from Tim Collart (EMODnet Secretariat): Will the Thematics be responsible for configuring metadata in the long term (like the static content).

• Answer: This is indeed the long-term goal

Question from Uffe Larsen (EMODnet Geology):

• They are planning to change the number of attributes for some layers. Do they need to wait with the deployment until MetaGIS is updated? How to coordinate this?
  • Answer: Let the central portal know through JIRA, and they will coordinate the updates to minimize downtime.
• There is a data cache, when will this be refreshed?
  • Answer: Usually every week, but it can be refreshed when an update is made.

Question from Graeme Duncan (EMODnet SBH): What if they need to update many layers at the same time, do they need to create a separate ticket for each?

• Answer: No in case there is a batched update this can be in one JIRA ticket.

Reminder from Joana Beja (VLIZ) to TWG that if there needs to be a press release/ news item, the information can be sent through JIRA to the COMMS team.

1. 3. Presentation on the collection of metrics for reporting.

Progress monitoring

(See documents TemplateEMODnet_QuarterlyReport_REV_202302, TemplateEMODnet_QuarterlyReport_REV_202302)

Nathalie Tonné (EMODnet Secretariat).

Collection of progress indicators as of January 2023 are collected by Europa Analytics (this is for the Thematics, ingestion will have to keep using the Grafana dashboard until further notice).

Narrative:

This will become shorter (in discussion with DG MARE/CINEA and approved)

• Section on events attended will be removed.
• Communication assets: will be focused on publications, do not include presentations, which are already mentioned in section 7.

Indicators Spreadsheet:

It will remain a document that collects information from the portals, together with stats from Europa Analytics. We will remove the indicators of the Grafana dashboard, replaced by indicator 5, which will be information from Europa Analytics, sent to the portals at the end of each quarter.

• You must add the graphs and some tables from the Europa Analytics report in the excel file.
• You will need to calculate trends (graphs from before and after trends in tables)

Comment from Chat. Antonio Novellino (EMODnet Physics): For Physics, while volume and coverage is ok to keep as it is, we are now not able to really track Indicator (1b).

Comment from Chat. Graeme Duncan (EMODnet SBH).

• We can only track the amount of requests/volume downloaded when it passes by our servers from JNCC.
• User feedback in interim/final reporting. I assume some of this will be captured through the JIRA service management, and lots may receive additional feedback via other avenues (emails etc).
• For the JIRA based feedback, will it be up to the thematic lot to extract this information out of JIRA, or will it be passed to us?
  • Conor Delaney (EMODnet Secretariat) replies yes, it is up to the thematic

Comment from Chat. Antonio Novellino (EMODnet Physics) to everyone: By the way, would be good to have a web tool to query for the views, the pdf option is not the smartest tool, moreover we need to wait we get it (sometimes it clashes with other duties)

• Conor Delaney (EMODnet Secretariat) replies, yes, eventually you will have access to the Europa Analytics. There are segments which filter the Stats to a specific thematic (the narrative part & use cases tagged to the thematic).

Comment from Carlos Cerezo (DG Mare): Illustrates the impact of the use of the Europa Analytics API used to configure the segments of the Thematics. The configuration of the segments will be done once per quarter at the end of the quarter. Impact on the thematic that new use cases will only be added to their segment filter at the end of the quarter.

Question from Alessandra Giorgetti (EMODnet Chemistry) and Dick Schaap (EMODnet Bathymetry) on indicator of data & product downloads. The Thematics cannot provide the part of the download statistics for products that are hosted in ERDDAP. These need to be filtered to specific Thematics and provided to the Thematics each quarter by the CP team.

• VLIZ answers that they will investigate this, but it will not be ready for this quarterly report.

ACTION 15: In the assessment, the secretariat should indicate when a part of portal’s products is offered through ERDDAP, as this may explain a decrease.

ACTION 16: CP team to investigate how to extract the statistics from ERDDAP.

6. Overview of helpdesk function

Overview of Helpdesk function

Nathalie Tonné (EMODnet Secretariat) presents.

As per the tender specification, email need to be answered within two working days. this applies to queries that now come in via the JIRA service desk.

Nathalie Tonné (EMODnet Secretariat) presents the workflow how user feedback is processed:

• When users send queries to secretariatemodnet [dot] ec [dot] europa [dot] eu (secretariat[at]emodnet[dot]ec[dot]europa[dot]eu) it comes into the JIRA service desk and is assigned to Nathalie Tonné.
• User gets an automatic email confirming receipt.
• Nathalie Tonné assign the ticket to the relevant thematic coordinator who can follow-up:
  • Respond internally: this will not be seen by the user but is for internal discussion.
  • Respond: this will send an email to the user.
• When ticket is resolved, you can mark it done (time until resolved is collected). This will send another email to user, marking that the ticket has been closed and will allow the user to rate the service.

Nathalie Tonné (EMODnet Secretariat) shows the coordinators that have access to the JIRA helpdesk. And explained that Biology/SBH & Physics have a helpdesk email, this is preferred over single people as user requests may be missed due to times off.

ACTION 17: If possible, the Thematics to provide a helpdesk email and inform the secretariat, so that they can configure it in JIRA and use it for the helpdesk.

Question from Dick Schaap (EMODnet Ingestion/Chemistry/Bathymetry): How to reach the overview screen with the helpdesk tickets.

• Nathalie Tonné (EMODnet Secretariat) explains how to reach it and will follow up with Dick.

Question. Dick Schaap (EMODnet Ingestion/Chemistry/Bathymetry) notes that receiving a helpdesk email is fixing the issue out of the system, can it not be assigned to a group? This is an impact on the licenses, that's why we are asking it to fix it though a shared account.

• Nathalie Tonné explains that some tickets are followed up through email, and in JIRA. This causes and issue as the user question is not tracked. You can send that email into the system by replying to secretariatemodnet [dot] ec [dot] europa [dot] eu (secretariat[at]emodnet[dot]ec[dot]europa[dot]eu) with the original user email in cc, which will then be tracked by JIRA.

Question. Marco Alba (EMODnet Physics) asked how you can make people watchers of a JIRA ticket.

• Nathalie Tonné (EMODnet Secretariat) demonstrates how to do this.

Question. Alessandra Giorgetti (EMODnet Chemistry): Agrees that creating a helpdesk email is a good approach and they will try to adapt it. She asks if she can identify all the tickets for chemistry.

• Nathalie Tonné (EMODnet Secretariat) answers that we will start assigning Thematics as components which can then be used as a filter. For now, you can filter on other fields (e.g., Assignee).
• Joana Beja (VLIZ) shows how to filter JIRA tickets.

Graeme Duncan (EMODnet SBH) suggests using components rather than assignees (as ticket could be reassigned). It would be good if Nathalie Tonné can set this as she assigns the ticket.

Question. Graeme Duncan (EMODnet SBH) asks can a thematic resolve a ticket?

• Conor Delaney (EMODnet Secretariat) replies yes.

Question. Graeme Duncan (EMODnet SBH) asks what about feature requests for the MapViewer?

• Conor Delaney (EMODnet Secretariat), In that case we create a new ticket in the "EMODnet" Jira project ticket, but link it to the ticket of the helpdesk, informing the users that the process will take some time.

Question. Graeme Duncan (EMODnet SBH) asks if it be linked to a knowledge base system?

• Conor Delaney (EMODnet Secretariat) We have not considered that, but we can investigate it.

7. Technical Discussion open floor (as suggested at 12^th TWG)

Technical insights/knowledge share or topics for discussion. To be suggested by thematic groups.

Joana Beja (VLIZ) gives some examples of technical issues that could be discussed in this slot (e.g., at the next technical working group meeting):

• Issue with cascading WFS service in SBH.
• Human Activities issue with creating NetCDF in Lambert projection.
  1. AOB

• Kate Larkin (EMODnet Secretariat) gives an update on the SC, which will be on 23 May 2023 back-to-back with the EMD in Brest:
  • Morning session: Closed SC meeting.
  • Afternoon session: Open session to TWG and MKEG.
• They will try to make it hybrid but would like to ask everyone to attend in person.
• Further information will follow.

8. Wrap up, actions, next meeting.

• Next Technical Working Group Meeting: format, date, and location (Secretariat & all).
  • Would be advisable to do an online TWG in the next meeting, but then do an in-person meeting linked with the JAMBOREE in Brussels.
  • Conor will send around a doodle (or EU Survey?) to decide on a date for this online meeting?

Annex I: List of Participants