Since the last KDE Itinerary summary two month have passed that saw the 19.08 KDE Application release, Akademy and more than 250 changes to KDE Itinerary and its underlying infrastructure. As usual, here are some of the highlights.

New Features

Support for attaching the original reservation or ticket documents to a reservation element in KDE Itinerary has been added. This has been often requested and makes a lot of sense, as our data model will never be able to capture every tiny detail. So you might still end up in situations where access to the original documents will be needed, and in that case it’s quite handy to have them right there in KDE Itinerary. This does not only work manually, but KDE Itinerary does this automatically with source PDF files already, and the KMail plug-in will also get that feature.

Access to documents in the context menu of the reservation details page in KDE Itinererary.
Access to documents in the context menu of the reservation details page
Document page for the current reservation in KDE Itinereary, allowing to open or delete documents.
Document page for the current reservation

There is now a new exchange file format that contains all elements for a reservation: the extracted JSON-LD data, PkPass files if available, and now also the source document(s). This has become necessary for the document support, to send a bundle of files to the app (e.g. via KDE Connect), without losing their association. While this is mostly an implementation detail, it also allowed the easy implementation of an import/export feature in KDE Itinerary to backup your data.

KPublicTransport can now process cancellation indicators and disruption notes, which in turn allows us to display this in the alternative connections and departure views in KDE Itinerary.

KDE Itinererary alternative connection page showing a cancelled journey and connection notes.
KDE Itinererary alternative connection page showing a cancelled journey and connection notes

Infrastructure Work

Plenty of things happened behind the scenes as well:

  • The extractor engine can now perform some of the data parsing and processing in a separate process. The benefits for robustness and ease of integration in 3rd party applications have been described in a dedicated post.

  • The extractor engine is no longer limited to reservation data but can now also process schema.org objects that represent reservable things. This is a generalization needed to consume data found by the browser integration.

  • Post-processing of the extracted data can now also resolve ISO 3166-1 alpha 3 and UIC country codes, which in turn improves determining timezones on input using such country codes.

  • All result models provided by KPublicTransport are now incrementally populated, so a single unresponsive backend can no longer delay the availability of results.

  • KPublicTransport can now also be used entirely from QML without the need to performing certain operations in C++ code, which considerably eases the integration in existing QML applications.

  • KPublicTransport received further improvements on its plausibility checks of the results received from the various online backends. Quite unexpectedly we found these results sometimes contain bizarre elements, such as European cities ending up with geographic coordinates in Antarctica or transfers between stations that would require you to move faster than the speed of sound, to name just two. Such results are now discarded.

Fixes & Improvements

As usual there’s also some noteworthy smaller improvements all over the place:

  • The alternative connection view in KDE Itinerary now shows an iconized summary of the journey in collapsed items, making it easier to locate the most convenient connection.
KDE Itinererary alternative connection page showing iconized summaries of journey segments.
Alternative connection page showing iconized journey summaries
  • The data extractors received a large number of improvements, particularly around European train operators. That’s made possible thanks to various donated samples, online and at Akademy.

  • The extractor engine, the app and the KMail plug-in got support for DataMatrix ticket tokens, in preparation for supporting Renfe tickets

  • The result aggregation and merging done by KPublicTransport saw a number of improvements too, e.g. regarding considering timezone information, and regarding better handling of location identifiers that could but are not guaranteed to contain international standard identifiers.

Performance wasn’t forgotten either of course:

  • There’s now a fast path for the common case of processing grayscale images in the barcode decoder.
  • Custom extractor loading was optimized to no longer require regular expression evaluations.

Development Tooling

Another area of focus in the past months has been further improving the development tools, in particular for custom extractors. Thanks to this KItinerary Workbench is now able to edit and create custom extractor scripts entirely from within its GUI, without requiring application restarts to reload the extractor meta data anymore, and without needing the previous hacks with symlinking the extractor sources to the install location, etc.

Editor for custom extractor meta data and extractor scripts in KItinerary Workbench.
KItinerary Workbench custom extractor editor

Additional new convenience features include source code navigation from script engine errors or script debug output, and further improved access to vendor specific data blocks in UIC 918.3 ticket tokens.

KPublicTransport also got a new feature to support development: The entire network communication can now be logged by setting the KPUBLICTRANSPORT_LOG_DIR environment variable to an empty directory. For each backend files will be created in there for each request, containing the request that triggered it, the resulting network communication and the result KPublicTransport parsed from the backend’s response.

Contribute

A big thanks to everyone who donated test data again, this continues to be essential for improving the data extraction.

If you want to help in other ways than donating test samples too, see our Phabricator workboard for what’s on the todo list, for coordinating work and for collecting ideas. For questions and suggestions, please feel free to join us on the KDE PIM mailing list or in the #kontact channel on Matrix or Freenode.