ECOMS 2016: Three days in Exeter

Last week I attended the European Climate Observations, Modelling and Services (ECOMS) at the MetOffice in Exeter. The conference itself lasted three days and for me personally it was the first time to pay Exeter a visit. The event was the last meeting for three EU projects, which in the days before had meetings on their own. All three projects are involved around seasonal and decadal climate predictions and cover different aspects of it. The projects were NACLIM, concentrating on the physical mechanisms and observations, SPECS, focussing on the modelling, and EUPORIAS, looking for the establishing of climate services. As such the topics and the background of the attending scientists were quite diverse, leading to interesting talks and conversations.

MetOffice in Exeter

MetOffice in Exeter

Continue reading

Why media reporting on discussion papers can lead to manipulated science

During this week discussions on twitter and on the blogs focused on a discussion paper by James Hansen et al in Earth System Dynamics Discussions. The paper forms part of a legal case in the US and basically states that the current and expected warming over the next decades is unprecedented since the last interglacial. In this context the Guardian has run an article on the paper. While they state that the science is not yet peer-reviewed, the authors have run a series of interviews and comments, which is usually happening only when a paper is actually published. As usual I refrain in this blog from commenting on climate politics, but as I have written my PhD on scientific publication processes I would like to focus here on the implications of the media scrutiny within the discussion paper phase of a scientific publication. Continue reading

Dames 2016: Conference report

Last week the conference on Data Analysis and Modeling in Earth Sciences took place just around the corner in Hamburg. As one of the smaller conferences it stretched over three days. Each day was divided into two parts: in the morning three keynote speakers had 45 Minutes to give a broad overview on their topics, while in the afternoon the talks lasted 30 Minutes.

The talks covered topics from Nonlinear Time Series Analysis to Big Data and few were on the applied side. Some went really deep into the theoretical background, which was good to make the long talking times interesting, and so were consequently quite mathematical. A main theme was also statistical parametrisations, a topic, which is often not so high on the agenda on the conferences I visited in the past. And as always, extremes were on the topics list, as it is currently one of the main themes on which people focus their research. A poster session on the first evening, conference dinner on the second and coffee and lunch breaks animated further conversations.

All in all it was an interesting conference. Having mathematicians beside those on the applied side, was a nice change to the usual separation. Of course it makes it sometimes hard to follow long talks, when they go into many details, but it was an interesting experience. The usual international 12 + 3 talks are tiring as well, but more due to the fast change of topics and the necessity to think into new talks avery quarte rof an hour. Longer formats give you the feeling, more like a lecture during your studies, that you understand more of the presented content. As such I liked it, but of course it just works when the number of participants is low. Myself had a poster in the poster session and it was the first time that I presented my current main work on the NAO. As such it was an important step towards the first publication to it, which I hope to submit until the end of the year.

The burden of maintaining an R-package

During my PhD I worked on Quality Assurance of Environmental Data and how to exchange quality information between scientists. I developed a concept for a possible workflow, which would help all scientists, data creators and re-users, for making data publications much more useful. One major foundation of this were quality tests, which I either taken from existing literature or developed anew.

Part of this work was the development of a proof-of-concept implementation of the methodologies. I used R, which is my prime language for quite a while, to design an as much as possible automisable test workflow. It was quite complex and in retrospect a bit too ambitious for real world applications. Anyway, as I prefer open science, I published it as an extension package for R in 2011: qat – Quality Assurance Toolkit.

The publication process was more challenging as anticipated. For each function, and my package had more than a hundred, a detailed help file was requested, which cost me at that time quite a while to create. I also wanted to add additional information, like an instruction manual, so that at least in theory it would have been possible to use the full functionality (like automatic plotting and saving of the test results) could be understood. Finally, when it was uploaded, I was happy and extended it until my PhD project came to an end.

Unfortunately, with this the work on the package has not stopped. R as a language is constantly changing, not really on the day-to-day tools, but in the background of the packages. New requirements come up now and then, usually associated with a deadline for package maintainers. What is quite simple to solve for small packages, can be a real challenge for complex ones like mine. I had to eliminate my instruction manual when the vignette system changed and created a dedicated website to have it still accessible. Also I had to replace packages I depend on, which is usually associated with quite a bit of change in the code.

All these changes are doable, but the big problems start with the requirement, that a newly uploaded package has to fulfil the current norms of the R packages. A package, which was fine a few months earlier has to change dramatically with the next update. This leads usually to a time problem, as each update needs therewith several days. So minor changes to the original code lead to a heavy workload. This lead to the situation, that I was not able to update it on time when the last deadline turned up and so my package went to archive. Half a year later I found some time and have now brought it back up to the CRAN network.

All in all, this workload is keeping me off to create new R packages. Making them would be feasible, but maintaining them is a pain. With these constant policy changing measures, R gets more and more out of fashion for heavy users and with it, it is in danger to lose out compared to other languages like python in teaching for the next generation of scientists. My personal hope is that future development will lead to a more stable policy on the package policy within R, so that more packages will be available also for the future. As things stand, I am happy to have my package up again, but when the next deadline will enter my mailbox, I will again have to evaluate the threatening workload, before I can afford to schedule a new release.

Massive ensemble paper background: What will the future bring?

In my final post on the background on the recently published paper, I would like to take a look into the future of this kind of research. Basically it highlights again what I have already written at different occasions, but putting it together in one post might make it more clear.

Palaeo-data on sea-level and its associated datasets are special in many regards. That is what I had written in my background post to the last paper and therefore several problems occur when these datasets are analysed. Therefore, as I have structured the problems into three fields within the paper I also like to do it here.

The datasets and their basic interpretation are the most dramatic point, where I expect the greatest steps forward in the next years. Some paper came out recently that highlight some problems, like the interpretation of coral datasets. We have to make steps forward to understand the combination of mixed datasets and this can only happen when future databases advance. This will be an interdisciplinary effort and so challenging for all involved.

The next field involved are the models. The analysis is currently done with simple models, which has its advantages and disadvantages. New developments are not expected immediately and so more the organisation of the development and sharing the results of the models will be a major issue in the imminent future. Also new ideas about the ice sheets and their simple modelling will be needed for similar approaches as we had used in this paper. Statistical modelling is fine up to a point, but there are shortcomings when it goes to the details.

The final field is the statistics. Handling sparse data with multidimensional, probably non-gaussian uncertainties has been shown as complicate. There needs to be new developments of statistical methodology, which are simple on the one side, so that every involved discipline can understand them, but also powerful enough to solve the problem. We tried in our paper the best to develop and use a new methodology to achieve that, but there are certainly different approaches possible. So creativity is needed to generate methodologies, which do not only deliver a value for the different interesting parameters, but also good and honest uncertainty estimates.

Only when these three fields develop further we can really expect to get forward with our insights into the sea-level of the last interglacial. It is not a development, which will happen quickly, but I am sure that the possible results are worth the efforts.

Massive ensemble paper background: What can we say now on the LIG sea-level?

After the new paper is out it is a good time to think about the current status on the main question it covered, the sea-level during the LIG. Usually I do not want to generalise too much in this field, as there is currently a lot going on, many papers are in preparation or have just been published and the paper we have just published was originally handed in one and a half years ago. Nevertheless, some comments on the current status might be of interest.

So the main question the most papers on this topic cover is: How high was the global mean sea-level during the last interglacial. There were some estimates in the past, but when you ask most people who work with this topic they will answer more than six metre  higher than today. That is of course an estimate with some uncertainty attached to it and currently most expect that it will not have been much higher than about nine metres than today. There are several reasons for this estimate, but at least we can say that we are quite sure that it was at least higher than present. From my understanding, geologists are quite certain that at least for some regions this is true and even when the data is sparse, meaning the number of data points low, it is very likely that this was also the case for the global mean. Whether it is 5, 6 or 10 metre higher is a more complicate question. It will still need more evaluation until we can make more certain statements.

Another question on this topic are the start point, end point and duration of the high stand. This question is very complex, as it depends on definitions and the problem that in many places only the highest point of sea-level over the duration of the LIG can be measured. That makes it very complex to say something definitive especially on the starting point. As such, our paper did not really made a statement on this, as it just shows that data from boreholes and from corals are currently not stating the same answer.

The last question everybody asks is the variability of the sea-level during the LIG. Was it just one big up and down or were there several phases with a glaciation phase in the middle. Or where there even more phases than two? Hard questions. The most reliable statements say that there are at least two phases, while from my perspective our paper shows that it is currently hard to make any statement basing on the data we used. But also here, new data might give us the chance to make better statements.

So there are still many questions to answer in this field and I hope the future, on which I will write in my last post on this topic, will bring many more insights into this field.

Massive ensemble paper background: Data assimilation with massive ensembles

Within the new paper we developed and modified a data assimilation scheme basing on simple models and up to a point Bayesian Statistics. In the last post I talked about the advantages and purposes of simple models and this time I would like to talk about their application.

As already talked about, we had a simple GIA model available, which was driven by a statistical ice sheet history creation process. From the literature, we had the guideline that the sea level over the past followed roughly the dO18 curve, but that high deviations from this in variation and values can be expected. As always in statistics there are several ways to perform a task, basing on different assumptions. To design a contrast to the existing literature, the focus was set to work with an ensemble based approach. Our main advantage here is that we get at the end individual realisations of the model run and can show individually how they perform compared to the observations.

The first step in this design process of the experiment is the question how to compare a model run to the observations. As there were several restrictions from the observational side (limited observations, large two-dimensional uncertainties etc.), we decided to combine Bayesian statistics with a sampling algorithm. The potential large number of outliers also required us to modify the classical Bayesian approach. As a consequence, we were able at that point to estimate for each realisation of a model run a probability.

In the following the experimental design was about a general strategy, how to create the different ensemble members so that they are not completely random. Even with the capability to be able to create a lot of runs, even realisations in the order of 10,000 runs are not sufficient to determine a result without a general strategy. This lead us to a modified form of a Sequential Importance Resampling Filter (SIRF). The SIRF uses a round base approach. In each round a number of model realisations are calculated (in our case 100) and afterwards evaluated. A predefined number of them (we used 10), the best performers of the round, are taken forward to the next and act as seeds for the new runs. As we wanted a time-wise determination of the sea-level, we chose the rounds in this dimension. Every couple of years (in important time phases like the LIG more often) a new round was started. In each the new ensembles branched from their seeds with anomaly time series for their future developments. Our setup required that we always calculate and evaluate full model runs. To prevent that very late observations drive our whole analysis, we restricted the number of observations taken into account for each round. All these procedures led to a system, where in every round, and with this at every time step of our analysis, the ensemble had the opportunity to choose new paths for the global ice sheets, deviating from the original dO18 curve.

As you can see above, there were many steps involved, which made the scheme quite complicate. It also demonstrate that standard statistics get to its limits here. Many assumptions are required, some simple and some tough ones, to generate a result. We tried to make these assumptions and our process as transparent as possible. As such, our individual realisations, basing on different model parameters and assumptions on the dO18 curve, show that it is hard to constrain the sea-level with the underlying datasets for the LIG. Of course we get a best ice-sheet history under our conditions, that is how our scheme is designed, but it is always important to evaluate whether the results we get out of our statistical analysis make sense (basically if assumptions hold). In our case we could say that there is a problem. It is hard to say whether it is the model, the observations or the statistics itself which make the largest bit of it, but the observations are the prime candidate. Reasons are shown in the paper together with much more information and discussions on the procedure and assumptions.