Which data source is better?

In the world of specialty crops, there is disagreement on how well weather-driven insect, disease, and frost prediction models actually perform. Dr. Dave Brown, former director of Washington State University’s AgWeatherNet spent years comparing different weather data sources and how those data affect the accuracy of common environmental models used by orchard growers. In this 20-minute webinar, he shares the surprising things he learned.

Decrease chances of crop damage with one simple practice

Find out how you can increase the accuracy of your predictive models and decrease frost, insect, and disease incidents by doing just one thing differently—improving the quality of your weather data. Discover:

• Microclimates: what are the conditions like inside a crop canopy versus outside?
• Virtual data vs. weather station data: Which is better?
• How do site-specific weather data vs. regional network data compare?
• How much does a small decrease in data quality affect the accuracy of your models?
• What’s the value of in-orchard measurements?
• What are some best practices for higher data quality?

Next steps

• Download slides
• Related webinar: Which weather station is right for you?
• Related article: Which weather monitoring system is right for you? 
• See the ATMOS 41 all-in-one weather station
• Request a ZENTRA Cloud live demo

Questions?

Our scientists have decades of experience helping researchers and growers measure the soil-plant-atmosphere continuum.

• Talk to an expert
• Request a quote

Transcript

BRAD NEWBOLD 0:09
All right. Hello, everyone. Welcome to Weather Data: Virtual, In-Field, or Regional Network—Does It Matter? Today’s presentation will be about 20 minutes, followed by about 10 minutes of Q&A with our presenter Dave Brown, who I’ll introduce in just a moment. But before we start, we do have a couple of housekeeping items. First, we want this webinar to be interactive. So we encourage you to submit any and all questions in the Questions pane. And we’ll be keeping track of those for the Q&A session toward the end. Second, if you want us to go back or repeat something you missed, don’t worry. We’ll be sending around a recording of the webinar via email within the next three to five business days. All right. With all of that out of the way, let’s get started.

BRAD NEWBOLD 0:49
Today we’ll hear from Dr. Dave Brown, who will discuss different weather sources, excuse me, weather data sources, and how they affect the accuracy of common environmental models. For 20 years as faculty member at Montana State University and Washington State University, Dave Brown pursued research on soil sensors, spatial data science, and digital agriculture. At both universities, he served in many leadership roles for major research projects, academic programs, and most recently as director of the WSU AgWeatherNet program. In this capacity, he hired and supervised a team of meteorologists who pursued research and extension activities focused on evaluating and improving the quality of weather data used for agricultural decisions. So without further ado, I’ll hand it over to Dave to get us started.

DAVE BROWN 1:38
Thanks, Brad, this is a great opportunity to sort of summarize some of the work I did as a faculty member and continue to pursue with METER and talk about in particular, you know, weather data as it applies to agriculture. So what kinds of weather data should growers be looking for for their decisions? And one of the things we need to first of all address is that the quality of the data and the type of data you need depends on the use. So that’s going to depend in part on are you interested in in pest management models? Are you interested in irrigation, frost, pesticide drift, are you just wanting to know when you need to deal with heat stress for your employees, but also what kind of crop? So if you’re growing a high value fruit crop, you have different needs and a lot of sort of in season decisions to make relative to someone who has extensive wheat, which a lower value per acre. And maybe you need a different kind of weather data for wheat than you do for cherries, for example. So with that sort of background, I’m going to, first of all talk about the weather sources in general, and then in particular, dive into virtual weather data, as it’s called.

DAVE BROWN 2:52
So we’re going to start with global weather models. These are actually models that have been developed over many decades to predict weather. And they integrate sort of satellite data and some ground observations and balloons and things of this nature to physically predict what’s happening in the atmosphere, not just at the surface, but all the way up. So we can see what’s happening in different layers of the atmosphere. And essentially, a lot of the weather data products that are available to growers now come from these physical models that are used to predict weather. But you can think of the current weather as time zero in those models. And this is what people often refer to as a virtual weather station. I’m not sure I like that term, because it really implies there’s something generating weather at a particular location, when in fact, there’s this global grid that you’re pulling data from. These are largely derived and run by governments and governmental agencies, because they do take a lot of resources to develop and run. There are some private versions, at least for parts of the world. But most of these commercial products that you might be getting on your phone app, and so on, a commercial company basically takes the output from this physical model. And they either combine multiple models, it’s fairly common to do that. And they provide some sort of a statistical interpolation and correction to the output of that model, and then deliver a grid to your phone, so you can put in longitude and it estimates the current weather, as well as the forecast going forward, based on this. So that’s one source of data.

DAVE BROWN 4:29
Another source of data that that people have traditionally used is you have some sort of a network of stations. This would be like AgWeatherNet in Washington State, but there’s mesonets in many states. In fact, most states have some sort of a Mesonet. Washington State has the largest in terms of number of stations. And these are intended to monitor regional weather. And so the key thing about these networks is not so much the equipment which can vary, you can have a very expensive tower, it’s getting 10 meter wind, and replicate, you know, precipitation buckets and so on, down to a lot of these mesonets actually use the ATMOS 41 that METER manufactures and something in between, like that AgWeatherNet station you see in the middle here. And so with with these networks here, the key thing is the location, they’re intended to be regionally representative, so they’re supposed to be on level open areas without obstructions, with grass cover, because that’s thought of as regionally representative. The nice thing about these measurements is that they have a long term record, you have fairly standard configurations to them, have a regular maintenance, and so they tend to be fairly high quality. But just in the very definition of a mesonet, they’re not intended to monitor the microclimate. And I can tell you that as AgWeatherNet director, we frequently had people asking to put a station next to their orchard or vineyard. And that actually isn’t the intention of these networks, they’re meant to serve a regional need. So if you want a site specific station, growers have been putting in their own stations for a while. I think in the past, these were quite difficult to maintain, and or they required more expertise to install than many growers had. But these days, they’re becoming less costly and easier to maintain. And so you’re seeing more and more growers wanting to get a weather their own site.

DAVE BROWN 6:26
Now there’s two different kinds of installations you can think about. One is a station that’s outside of the canopy that’s intended to capture that local microclimate. And the other is inside the canopy. So it’s intended to capture both the microclimate and the management effects like your irrigation, any kind of evaporative cooling or wind machines and so on, they will be reflected in the measurements made within the canopy. There’s quite a difference of between what you measure outside and inside a canopy particularly for something like an apple orchard. High density apple orchards we studied and this is funded by the Washington Fruit Tree Research Commission with Joseph Broadneck, and Lee Kalcsits and Lav Khot has recently come on board with that project. And we looked at inside versus outside these high density apple orchards. I’m just going to show you four. There’s actually I think, 13 sites now that were making these measurements, but what you see is overall and this is pretty consistent, is that that in season when there are leaves out and so on that it is cooler inside the canopy than outside the canopy. This isn’t too surprising, because you have transpiration from the trees, you have irrigation, which could be throwing water around, you can have evaporative cooling. But also we noticed that it seems like cold air is trapped in this orchard at night. So we get cooler conditions at night when there are inversions, stronger inversions. So you see, this is the summer of 2020 here and it’s a little over two degrees Fahrenheit, on average, this is sort of the average effect here. And then we had this really hot summer, there’s a lot more evaporative cooling and it gets down as much as four degrees Fahrenheit as net effect. This will have a big effect on your crop models and your SRU pest models, your disease models, even your blue model and so on. Relative humidity also reflects this. It’s wetter inside the canopy than outside of canopy. And wind speed, we can see that basically wind is blocked inside the canopy when the leaves are out. And so in both summers, we see that it’s systematically lower wind speeds with inside the canopy. The one exception is we see here in late March and we see that we that get a little bit windier inside the canopy. And that’s likely due to wind machines protecting against frost. So if you’re making decisions about pest and disease management, at a minimum, take into account these effects of the canopy on the actual conditions versus what’s outside. Or better yet, put a sensor inside your canopy to capture that directly. We do actually as a company recommend that you put your main weather station outside of the canopy and that might sound illogical but the reasons are that, one, there’s a couple of measurements that we need outside of the canopy. For drift regulations, you need to make wind measurements outside of the canopy. And secondly, evapotranspiration needs to be made outside the canopy to sort of capture the effect of the total environment on that canopy. Second main reason is wear and tear. This is a valuable piece of equipment and anything you put inside the canopy is vulnerable to damage. But what I think from a scientific perspective is particularly interesting is that if you want to know the value of your management what you’re doing, let’s say turning on your wind machine or running an evaporative cooling, you need a reference measurement. So having something outside the canopy to see the effect of what you’re doing is really valuable. And lastly, we’re moving towards a system, starting with METER and with other places that you can train the weather forecast on data coming from the station. So you get very site specific forecast. And to do that, you really want a station that’s not contaminated by your management. So for all those reasons, we recommend putting a station outside the canopy and then having lower cost temperature, relative humidity sensors inside the canopy.

DAVE BROWN 10:23
So net effects, and this is just one example. And of course, these will always be different depending on on specific location. But this is looking at an orchard here on Fir road. And there’s a weather station inside the canopy, a weather station outside of the canopy. And there’s two neighboring AgWeatherNet stations here not too far away, a few miles away. And we also pulled in virtual weather data from DarkSky to do a comparison there and looked at just growing degree accumulations, base 41. And look at 1000, just because it shows up nice on the graph. But that’s also close to when you might want to address Western cherry fruit fly. But for this analysis, we can see that the optimal curve here is the green line. That’s the actual conditions inside the canopy. And we see that the Ringold station here actually lines up really well with that. And that’s because actually the site at Ringold is actually cooler. But then because of the orchard effect, sort of two wrongs make a right and it ends up lining up. But then if we look at the Pasco station, it’s off by 15 days here. The DarkSky, this accumulation is off by eight days, and station outside the canopy is off by five days. Now, the weather stations, the nearest weather station, that can be off in different directions, it really depends on location and conditions. Same with DarkS ky that can be off in both directions, it just happens to be off in this direction this time. So one of them is systematic, the effect of the canopy, the rest are somewhat random.

DAVE BROWN 11:59
So I want to now dive into the virtual weather data and evaluating that because I think this is something that, on one hand is really appealing to growers, because it’s a low cost source of data, it comes from model, you don’t have to put in a station or maintain a station, it always arrives. So back when I was director of AgWeatherNet, we initiated a study again with Joseph Broadneck, my postdoc at the time, and we wanted to compare DarkSky to what we were measuring at our stations. And these are all Campbell Scientific stations with replicated air temperature sensors. So we’re able to screen these stations and make sure we had really high quality data coming from the stations, a good reference 156 stations, eight years. And I will say our point isn’t to disparage DarkSky, we chose this option, in part because it’s one of the best products out there in terms of this virtual weather station data. And we compare daily high and low temperatures. And this comes out to almost half a million daily comparisons when you add in all those stations and years and days. So quite extensive here. And if we look at the distribution of errors for the DarkSky data, relative to the ground truth, we can see that that the high temperature actually isn’t too bad. Most days fall plus or minus a couple of degrees, or just for perspective plus or minus a couple of degrees is not a great weather station, that’s probably a fairly inexpensive, not so great weather station. We aim for less than a degree Fahrenheit error on a weather station. And then, but you certainly get a fraction here that are quite large in their errors. And then we come to nighttime and we have these huge errors. And the reason for this reason why these models perform so poorly at night is that they really make no attempt to capture cold air flows at night and your local topography. These are global models and they can’t model all the details of your specific site. Since models are even though you can give them a specific latitude and longitude, they are not site specific, and that they’re not modeling the specific conditions at your orchard or your vineyard. So they have a warm bias overall one degree Fahrenheit, a very large spread. And the warm bias comes from the fact that on the coldest nights, the models fail to capture that and they come in very warm. So on the night when it’s unusually cold, they’re going to miss by quite a bit. That’s when you see these errors. So you certainly never want to use a model like this to make frost decisions. You also want to be very careful about something like a pollen tube where you’re making sort of daily accumulations of heat. You can maybe use them on longer accumulations where your errors average out if there’s no bias. B ut anything where short term data is required, these errors are just much too large. And then we also looked at systematic site bias. So we took a site where there was a station and looked at that accumulated bias over eight years. And you look at the high bias and there is some bias there. It can be up to two degrees, not so bad. But if you look at the low bias, some of these sites have some enormous errors, systematic bias at some of these sites. So that’s going to cause a real problem even for something like growing degree days, when you see biases that are this large. The final sort of kicker to this as these biases are not constant, they can change from year to year. So the physical models themselves are updated from time to time. As they try to improve the models they’re updated. And then the algorithms, these commercial companies use to sort of process the output of the physical model, those are also updated. They’re constantly trying to improve their algorithms. And it might improve overall the performance but might do some really weird things at your particular site. This is Sakuma station, this is one of the worst, Walla Walla was also particularly bad. So there’s some sites that really jumped around from year to year in terms of the direction and the magnitude of the bias. And so the effect of this, this is drying degree day, 50 degree Fahrenheit base from January 1. And this is meant to sort of 375 growing degree days and 1375 growing degree days, sort of a early season codling moth model and mid season codling moth model. Total combined, because we looked at each year separately, so 1248 station-years. So three quarters of the sites had what we consider to be a bad year, where the first date there, 375 growing degree days was off by at least five days, calendar days, at least once in 8years. So just because the distribution looks like, well, most of the time, it’s not off by that much. The problem is, it’s not the same stations every year, or the same sites every year that are off. That can change. And that’s going to be a real problem for a lot of growers. Now, by the time you come to mid season, of course, the errors get quite large, and really be difficult to use this at that point. So these biases are a problem. So I kind of want to talk about how growers actually use weather station data. And most growers I’ve talked to do not use the regional to AgWeatherNet Station directly. They know from experience because they track the performance of this over time, and relative what they see in the orchard that maybe their orchard is two degrees cooler on average or three degrees warmer or whatever that offset is and then they do a mental correction knowing that all right, if that weather station says my spray date is, you know, June 10, I’m going to move that and I know that it’s actually June 3 for me, because I know I have to apply this— I correct. I have a mental model that I can use to correct that data from the regional weather station, which doesn’t quite capture what’s happening in my orchard. Now, if you’re using virtual weather data, there’s two problems. Number one right up front, unlike the regional weather station, you can’t anticipate what kind of a bias there’s going to be because it’s not like it’s at a higher elevation, or it’s in a cold air draw or anything like that, that you can kind of anticipate what the bias would be or how far away it is. So you’re a bit blind going in. And second thing is I sort of liken this to the game Marco Polo. When you play that as a kid, you say Marco and someone shouts out Polo, and then you start to navigate towards him, you’re starting to dial it in, and then they you say Marco and they say Polo, and you get closer. And then you say Marco and you don’t hear anything, and then you say Marco and then you hear Polo from the other side of the field, the other side of the pool, because they’ve just swim underwater to a new location. And if you’re using virtual data, you are kind of blind to those biases. So you’re going to need to check it every year, keep making sure you know what the difference is between the virtual weather data and your actual orchard. So the bigger picture in my mind, and this is a longer conversation, we don’t have time for a lot of depth here, is that ultimately there is a lot of value in these global models and the grids that come out of them. And in fact, we use them at METER in combination with station data. And there’s a lot of value in these regional networks that have these longer term records, there’s things you can do with a long stable record that you can’t do with a grower who just puts in a station this year or maybe last year, or maybe they move the station because of some operational issue. And so they tend to have shorter records, and not as uncontaminated in a way. And there’s ways that you can use all three of these data sources together to generate the best possible predictions for pest models and frost and so on for a grower. So it really shouldn’t be seen as a competition. These should be seen as how can we use all these different sources of weather data the best way we can, and part of that might be if you don’t have, maybe it doesn’t make economic sense to put in your own station, there can be ways you combine virtual data with a regional network to interpolate your orchard in way that’s better than the virtual alone or the regional station network alone. And that’s something actually that AgWeatherNet has funding to pursue. Because I helped acquire that funding. So with that, I will take any questions.

BRAD NEWBOLD 20:18
All right. Thank you, Dave. And yeah, we’d like to take, you know, the next 10 minutes or so for questions. We do have several questions that have been submitted already. If you have any questions, now’s the time, submit them in the Questions pane. And we’ll try to get to as many as we can, before we finish here. If we don’t get to your questions, don’t worry. We do have them recorded. And Dave, or somebody else from our METER team of experts here will be able to get back to you via email to help answer your question directly. So feel free to submit any and all questions that you have. And we’ll try to get to as many as we can. All right. So let’s see. This first one here. Dave, WMO, and NWS have standards for the location and distances for weather stations and their sensor locations. Is there any standards for midionet and or microclimate stations? Our technicians have seen all sorts of small stations and different scenarios, even on top of metal roofs.

DAVE BROWN 21:23
Ideally, you don’t have absolute standards the way you do for the regional networks for a local site. But you try and do the best job you can to make sure that station that’s outside of your canopy is not affected by wind machines or evaporative cooling or irrigation or things of that nature, that to your best ability, and it’s not perfect, and you try to not have it over something like concrete or asphalt that get really hot during the day. And we try to at least have it over dirt and maybe some grass cover if you can manage it. But you do the best you can, knowing that it’s not ideal. Inside the canopy, we really recommend that that bottom wire— and an apple orchard, you’re really looking at kind of the bottom wire because you’re interested in things like dew and frost and lower in the canopy. Those are, that’s probably the better location to capture those. But you’re right that there’s not standards. Do the best you can.

BRAD NEWBOLD 22:24
Alright, next question here. What are the sources of ground truth data that they’re using to refine these models?

DAVE BROWN 22:35
Well, these models, one thing they do use is the Mesonets. So all these Mesonets are compensated by NOAA to feed their data into the National Weather Service, they sort of do all this assembly, and there are stations that are maintained directly by NOAA, there’s airport station data, it’s pretty important as well. So there’s a large number of sources of data that feed into these models. A lot of that isn’t necessarily used to build the model so much as to check it. And one of the things you have to keep in mind is that the purposes of these models are they’re trying to predict what’s happening all the way up in the atmosphere. And so surface data alone isn’t the sole focus. And so they depend really heavily on satellite data, and balloons and things like that, to get the data they need for these models. That answer it?

BRAD NEWBOLD 23:29
Sure. Another question, what about installing weather stations like METER’s ATMOS 41, you know, a half meter to a meter above the trees, so the reference surface is the top of the tree canopy?

DAVE BROWN 23:46
That’s something that some people do. One of the issues with that is that your wind speed is certainly going to be higher up there and your temperature. When you have an inversion, it’s going to be substantially warmer up there. And so the ideal surface measurement for air temperature, and this is how all the these models are calibrated and developed, is that one and a half to two meter, ideally, sort of one and a half meter height above ground. And so if you’re going to use that station to develop a site specific forecast, there’s a bit of disconnect between, you know, where your station is placed versus how the model was developed.

BRAD NEWBOLD 24:27
All right. This next one, would it be possible through modeling to obtain values of physical parameters such as temperature and humidity, from inside the canopy, using data from a station outside the canopy, for instance, you know, 300 meters away? For example, do you know any model that is able to predict the temperature 30 centimeters above the ground inside a potato field, using only data from a reliable weather station that is 300 meters from this field?

DAVE BROWN 24:56
I think you should be able to do it, it’d be empirical and you know, there’s differences in how you irrigate and the soils and, you know, crop stage and things like that that would have an effect. And, but I do think that you could probably empirically get something reasonable to anticipate what that might be. And there might be some published I just haven’t, you know, dug for that.

BRAD NEWBOLD 25:21
All right. You’ve talked about the current state of the science, in your opinion, will virtual weather improve substantially to deal with what you’ve shown, or what we always need local weather stations for accurate forecasts?

DAVE BROWN 25:39
Certainly, in the next decade or so I don’t anticipate the virtual weather data, being capable of addressing this issue and being resolved enough to overcome these problems. And in the part that you have to think about who is paying the bills for this virtual weather data. And largely, it’s things like, you know, renewable energy winds for wind farms, wind speed, and wind farms. It’s things like transportation sector, aviation, marine, and so on. And so their needs are a bit different than agriculture. And actually, and then the modelers themselves are heavily focused on things like extreme events, hurricanes, and tornadoes, and so on, storms that do a lot of damage, hail storms. And so trying to really nail in this really very site specific surface conditions, that’s not really the focus, and it would be pretty hard to do, given this current, you know, sort of framework we’re working in with these models.

BRAD NEWBOLD 26:41
Alright, I think we’ve got time for one more question here. And this is similar to ones that we’ve answered already. Is it better to leave a station inside a fruit orchard for more reliable GDD or ET data?

DAVE BROWN 26:56
Yes. So there’s different answers to those, there’s two questions there. So for growing degree days, you do get better data inside the orchard. But again, we recommend that you actually just put a temp RH sensor in there for that purpose. For evapotranspiration, you get better data outside the canopy, because evapotranspiration is meant to predict what the canopy as a whole does. And so you really need the wind speed, and the solar radiation and temperature outside of the canopy to sort of estimate the water loss from that canopy, and in fact, we’ve seen that when people put a weather station inside their canopy, the evapotranspiration values are far less, way way way off, actually.

BRAD NEWBOLD 27:42
All right. I think we’ve got time for another question. I lied. We can squeeze a couple more in here. Let’s see. How do I know if the weather prediction I have for my local area is based on virtual weather or if there’s actually a weather station nearby?

DAVE BROWN 28:07
That’s a good question. I mean, if you use something like Weather Underground, you can actually see the individual stations that feed into their system. But, you know, most weather apps that you’re going to use are using some kind of virtual— now keep in mind that virtual data they’re using is probably getting some kind of correction from the weather stations in the area. So if you’re in Seattle area, or somewhere like that, where there’s a lot of stations, you know that those stations are being used to correct that grid. And so it is, in some ways, kind of a hybrid. But most of these apps now are— because they want to make sure there’s always data flowing, they don’t want to worry about a weather station going down or having errors. And also that you notice all these apps, you know, have a forecast, well, that’s the model. And so to have the current flow right into the forecast, it makes sense to run it off of a model.

BRAD NEWBOLD 28:59
Okay, all right. This is going to be our final question. And again, thank you for everybody who submitted questions. Again, there are several questions that we do not have time to get to. Feel free to to pop in any more questions that you have before we finish here with this final question, and we’ll get back to you via email. All right. So this final question here, Dave, can we say with enough confidence that the local measurements for instance, in the orchards are the most relevant for modeling pests and diseases etc?

DAVE BROWN 29:34
Can we say with confidence are the most relevant? You know, it probably depends a certain extent on how the model’s developed. If the model was developed using stations that were outside of the canopy, then you want to be careful of using data from inside the canopy to run the model. But if you think about the coddling moth, that actually was done in chambers and greenhouses and so on, so that that’s fundamentally based on the actual temperature. So you really want the most accurate temperature for coddling moth. For something like fireblight, what I would say is really one of the key things with fireblight is a wetness event. Not so much the heat unit, I mean, heat units give you an idea of whether you’re at risk of fireblight. But that wetness event and those wetness events tend to be very local, so yes, you really do want to know if there was a dew event or something in a certain part of your canopy. And that’s getting most accurate. So I can’t say for every model— depends on how they’re developed. But for a lot of the key models, this really is the best data.

BRAD NEWBOLD 30:32
All right. Thank you again, Dave. That’s gonna wrap it up for us today. Thank you again, everybody in the audience for joining us. And we hope that you enjoyed this discussion as much as we did. Thanks again for all of your great questions. And like I mentioned before, several times, we will be able to get back to those questions that didn’t get answered live here today. Also, please consider answering the short survey that will appear after the webinar is finished, just to let us know what types of webinars you’d like to see in the future. And for more information on what you’ve seen and heard today, please visit us at metergroup.com. Finally, look for the recording of today’s webinar in your email. And stay tuned for future METER webinars. Thanks again, stay safe, have a great day.