Previously, we presented a picture showing that the magnetometer readings experienced a sudden drop in early 2019. Some may have interpreted that as a “collapse of the magnetic field.” However, we have determined the actual cause.
We found a strong correlation between satellite altitude and the magnetometer reading, so we used some simple AI to remove the effect of the changes in altitude, which explained all of the “apparent drop.” The blue line shows the altitude of the satellite. The red line is a smoothed reading of the raw magnetometer field strength data (block dots are raw.) As the satellites get further from the earth, the magnitude drops. The black line represents a corrected magnitude after subtracting the influence of the spacecraft’s height, which yields a relatively stable magnetic field through the period.
This morning, we produced this chart of the average strength of the Earth’s magnetic field. The blue line is linear regression and the red is a smoothing function. The view gets more fascinating with every addition of data. We are still not in a place to rigorously analyze and validate the apparent sudden drop, but my intuition is telling me it’s part of a pattern. Since the code updates aren’t yet complete, we’re using the opportunity to decode another batch of files.
Reprocessing for 2019 through a week ago is complete. It took a week because after three days – when it was nearly done – a Windows update came in and rebooted my machine. We lost all the data and had to start over.
It appears as the Earth’s electromagnetic field has been increasing of late, roughly for the last three years. However, early 2019 data looks like it may contain an error. We’ll have to do some forensic analysis, but potentially, the field may have been stepping down relatively quickly. You can even see why someone might argue that the trend has been down over the last three years (which is disingenuous in my view.)
Keep in mind that our interpretation of the data could change rapidly.
Below is an animation of the first three months of 2021. We’re no longer building the animation as we structure new data so that we can get the base data sooner. Later, we’ll be able to create animations in a parallel process.
We’re currently reprocessing the entire dataset, but this short clip shows that we’re handling nulls correctly now. We’re also going with a different color pallet. I’ve added more data, like the field vector, which I’m not yet utilizing. I have many more diagnostic processes. We’re now re-downloading files that got corrupted, and now we have a more complete file set and more.
There is also a new dataset on the way. It’s more radial on a spherical surface. However, it’s also more challenging to represent 2D graphically – but that will happen. This second study will give us perspective on the previous results.
I have a parameter that adjusts the size of the blue area, and there’s nothing like seeing what you are doing so I created the above graphic. The red dots are my aggregation nodes. For each red dot, there is a blue area. This blue area belongs to the red dot at x0, y0, z1. The aggregation process will average the magnetic field strength over the earth’s surface within the blue area for each red dot.
And in other news… I lost a bunch of code. Essentially I have to re-write the code that produces the above graphic and the data structure that averages the field strength at each red dot.
This is the Earth’s magnetic field strength for March 2019 by day, taken from the Swarm satellite array. The image is the Earth’s surface broken into “equal-area” parts. Equal areas allow us to calculate a general average field strength despite the Swarm satellites spending vastly more time at the poles.
We determined the bad frames came from improper null handling, which we already fixed in the code.
I’m seeing a slight uptrend for the global mean strength for the Earth’s magnetic field during March 2019, but it’s not significant. 34090 to 34130 is only a 0.11% difference.
All the latest files have been downloaded. Now we have to decode the – almost decade’s worth of – data, which is estimated to take two weeks! I’m using a Mathematica script because the files are encoded in an obscure NASA file format. Then we need to dimensionalize the data.
After consulting with various engineers and scientists who understand satellite data, I believe we have the data model needed to get to the heart of the matter fairly quickly. The raw sensor data will have some data normalization problems, but we’re probably anticipating the bulk of them. Apparently, I have two weeks to write the modeling code 🙂
I wonder if there was a magnetic disturbance associated with COVID. And I wonder if when I dream about the data, I’ll hear Earth’s thoughts.
I’ve decided to use my citizen scientist skills to look at the Swarm satellite magnetometer data and determine the rate the Earth’s magnetic field is dropping. Ben Davidson of Suspicious0bservers has made it a daily ritual to fearmonger about the pace the field is collapsing, but I can’t find any up-to-date information on this.
Currently, we’re still downloading and decoding the data. However, we’ll soon be seeing what’s really going on with that magnetic field of ours!