metaf2xml: parse and decode METAR, TAF, SYNOP, BUOY, AMDAR and write data as XML
Description
metaf2xml can download, parse and decode aviation routine weather reports
(METAR, SPECI, SAO), aerodrome forecasts (TAF), synoptic observations
(SYNOP), observations from buoys (BUOY) and meteorological reports from
aircrafts (AMDAR). Data can also be taken from decoded BUFR messages.
The extracted data can be written as XML or passed to a user-defined
function (all done in Perl).
It also provides XSLT style sheets to convert the XML to plain language
(text, HTML), or XML with different schemas. A web-based (CGI) user
interface can be used to download and display up-to-date weather data from
NOAA, Ogimet, and other sources.
The latest version ()
is 2.10
(description of changes).
It requires Perl 5.14.4+ which
is available for
Unix/Linux, macOS, and Windows.
The source code is available as
TAR or
ZIP archive and in
Git.
Various
installable packages
are also available.
Background
METAR and SPECI (weather observation) and TAF (weather forecast)
information is essential for flight planning and in-flight decisions (real
flight and simulation).
It is regularly compiled by meteorologists for more than 6200 airports
worldwide and distributed via reliable, non-public networks.
SYNOP information is collected by more than 7600 manned and unmanned
meteorological stations and more than 2500 mobile stations around the
world and is used for weather forecasting and climatic statistics. The
format of the original messages is abbreviated, some items are coded.
Here are some METAR, TAF, and SYNOP messages from Rio (SBGL, 83746), New
York (KJFK, 74486), and Tokyo (RJTT, 47662) and messages from nearby
buoys (as alphanumeric (TAC) and decoded from BUFR):
The format of the messages (and a lot more about observing and reporting
weather) is specified in WMO Manual No. 306 or in Annex 3 to
the Convention on International Civil Aviation; some countries deviate
from that standard or extend it, some have published documentation (e.g.
Federal Meteorological Handbook No. 1 and No. 2
for the USA).
Especially the format of the remarks for METAR and TAF messages, or the coding
of sections 3 and 5 for SYNOP messages depend on the country or even the
observing station.
metaf2xml can be used for flight simulations or by private pilot (students) who
want to extend their knowledge about the message formats, and everyone who just
wants to know the current or forecast weather at some station in plain language
(see examples below).
Why another METAR/TAF parser?
There are other projects that deal with METAR and TAF messages, e.g.:
the C library MDSP from the NOAA/NWS,
on SourceForge
(inactive),
on GitHub,
an MS Windows binary (runs with Wine) in a ZIP archive from
MetarWeather,
the Aviation Weather Center
of the NOAA translates METAR and TAF messages (most elements) from their
database,
the web site
Ogimet provides graphical and
text-based interfaces to up-to-date and older METAR, TAF, SYNOP and BUOY
data (collected from NOAA),
the web site
Meteomanz.com provides graphical and
text-based interfaces to up-to-date and older SYNOP data; most parts of the
messages are also translated to plain language (English, Spanish),
Allmetsat
provides a graphical interface to METAR and TAF data (source: NOAA/IWS); the
most relevant parts of the messages are also translated to plain language.
But somehow I always missed a feature I liked to have.
Features of metaf2xml
3.524
processing of messages
The parser module
(man page)
accepts (= processes) or tolerates (= marks as error) many extensions to the
standard; they are marked with "EXTENSION" in the source code.
Many common minor errors in the format are corrected before parsing, more
combinations of weather phenomenons are allowed, missing information is
recognised in many cases, too.
All remarks are processed, some of them are interpreted depending on the
issuing country or station, e.g. 'MON INVIS VAL FOSCHIA' and QUL/QUK (Italy),
'JP' (some Caribbean countries), 'YLO1' (military), everything from the
Federal Meteorological Handbook No. 1 and the Air Force
Manual 15-111 (19.12.2003), 'RSNK' and 'LAG PK' (data from other
observing stations), and many more.
Currently, more than 80 types of METAR remarks are recognised, excluding trivial
keywords (see also the
DTD).
This module can write the data as XML.
Alternatively, a callback function can be registered which is passed step
by step the same data in the same order as for the XML output.
The data contains solely information from the original message, with 2
exceptions: the relative humidity (actually 4 different values from 4
different formulas) and the ceiling (if there is one); these 2 items are
cumbersome to compute with XSLT.
For each data element, the corresponding text from the message is
provided.
The intention is to gain as much information as possible from a message
without interpreting parts of it wrongly, and that no parsing of the output
data should be necessary.
source of messages
Messages can be specified simply as text.
Additionally, the script metaf.pl
(man page)
can be used from the command line or as CGI script to download and display
in plain language up-to-date weather data from Internet servers, an SQL
database, or a local file.
metaf.pl uses the module src2raw.pm
(man page)
to extract correctly formatted messages from the files provided by public
servers.
display of XML file
The project also provides XSLT style sheets to convert the XML to plain language
as text or HTML in
English,
German, and
Spanish.
The HTML output displays each message in a separate table.
The leftmost column contains the original text for each element of the message,
the top row contains the original message (see also project screenshots or
examples below).
To convert the content to text or HTML, the program "xsltproc" is used, but
others that work similar shouldn't be a problem.
There are also XSLT style sheets to create a summary or overview: only some
items of each message are shown in an abbreviated format, one line per
message.
command line interface
The script metaf2xml.pl
(man page)
reads messages, processes them, and writes the XML data to
standard output.
It can serve as input for "xsltproc" or similar programs to get HTML or text.
The script metaf.pl
(man page)
can be used to fetch up-to-date messages from other sources.
web interface
Ideally, any XML capable browser should be able to process the XML with the XSLT
style sheet specified in the file, and display the formatted result, but not all
browsers are XML capable.
The script metaf.pl
(man page)
provides an interface using either XML/XSLT or pure HTML.
The XML data can be seen by viewing the source of the web page if "XML"
was selected as output format.
source code is available and GPL
Everybody who wants to contribute is invited to do so.
The best starting point is the
ticket system.
To try it out
The latest version of metaf2xml can by tried out at SourceForge.
However, the demo version cannot provide up-to-date messages for stations,
as SourceForge does not allow outgoing connections to the internet. The
messages are fetched from a database on the SourceForge server instead. The
database is updated from time to time, only. Currently, it has data for
.
With the
demo version with graphics
(uses Ajax), the stations are marked with a dot on a map.
The map can be viewed in different enlargements and can be moved by click and
drag.
The requested messages are displayed beneath the map, the latest one on top.
The list of stations is created dynamically from the same database.
With the
text-based demo version,
weather messages can be entered.
If an option with "id(s)" is chosen, station id(s) can be entered, too.
Sources for weather data
Several organisations and providers offer public access to up-to-date and older
weather data, e.g.:
The result of the parsing of the above example messages can be found in
example.xml
(XML without XSLT style sheet reference).
Using the XSLT style sheets from the project, the output in plain language
(here: as HTML, summary first, detailed below) looks as follows: