Big data vs. accurate data
/“Shit in = Shit out” the first law of numerical simulation
In meteorological weather prediction, every student of computational fluid dynamics (CFD) knows that one cannot obtain any relevant levels of accuracy without accurate boundary conditions and initial conditions (input data), which equates to accurate, surface, near surface and upper air meteorological data.
Above every square meter of ground there is over 7,000 kg of air in the troposphere alone. Troposphere is where weather happens. It also contains 99 % of all atmospheric water vapor. It also contains most of the mass and energy of the whole atmosphere. [1]
The acceptable magnitude of error
Relevant weather forecasts depend on accurate input data. Garbage-in = garbage-out.
The amount of energy trapped in the Troposphere is striking. On a 37 °C summer day with 95 %RH humidity just one kilogram of air contains about 137 kJ of energy. A +3 °C error in air temperature reading due to inadequate solar shielding [2] will erroneously increase this figure a whopping 16 % to about 159 kJ/kg. A large 5 %RH humidity deviation will only cause one fourth the error. Applying even a fraction of this error to boundary conditions of a CFD weather model of the local troposphere will lead to uncorrectable errors in numerical simulation and erroneous results and forecasts.
Validation counts
Having worked on wing design for a company which arguably makes the most efficient airplanes in the world, I had a first-hand look at complexity of numerical fluid simulations and their susceptibility to error. Results from numerical weather prediction models require validation. Just like airplane companies and even Formula 1 use wind tunnels to validate their CFD simulations, meteorologists validate with surface, near surface and upper air meteorological data. Yet there is one big difference. The meteorological weather prediction models have much more complex inputs and boundary conditions, thus more chance for error. Unfortunately, false validation seems to be a trend as many meteorologists are love to claim high weather forecast accuracies. I think we all intuitively realize the reality in weather critical situations is quite different.
Want to know more?
Read more why "Garbage in = Garbage out" is the most important sentence a modern CEO needs to hear and about the cloud of difference data quality will make for your next "Big-Data" cloud platform in the 2019 Varysian Guide to be distributed at the Meteorological Technology World Expo 2019 in Geneva.
Rererences
[1] "Troposphere". Concise Encyclopedia of Science & Technology. McGraw-Hill. 1984
[2] METEOMET: An experimental method for evaluation of the snow albedo effect on near surface air temperature measurements, by Chiara Musacchio Graziano Coppa Andrea Merlone
About the author
Jan Barani is the inventor of the helical MeteoShield® Professional and the CTO of BARANI DESIGN Technologies which brought to the professional meteorological world the 1st micro-weather station, called the MeteoHelix® IoT Pro, to be able to meet WMO guidelines for precision meteorological measurement in all weather conditions and all climates.