Heisenberg: Surf the Wave-Particle (Confidence Tradeoff)

The interplay of politics and the media with music and culture
FishbellykanakaDude
Posts: 1313
Joined: Tue Jan 09, 2018 8:07 pm

Heisenberg: Surf the Wave-Particle (Confidence Tradeoff)

Post by FishbellykanakaDude »

https://www.youtube.com/watch?v=MBnnXbOM5S4

..how would you use the (fast) Fourier Transform to up the confidence of a "systems prediction's" two primary characteristics?

Help John narrow down the timing of his "predictions", as he seems to have it's "tradeoff characteristic", it's inevitability, reasonably well in hand.

Timing is high frequency. Inevitability is low frequency.

Ready,.. set,.. GO! Change the word, dude!


:) Aloha hoaloha! <shaka nui!>

Heisenberg
Posts: 71
Joined: Mon Feb 19, 2018 2:52 pm

Re: Heisenberg: Surf the Wave-Particle (Confidence Tradeoff)

Post by Heisenberg »

FT could be interesting if used locally (for each identity group) to see if the 80 year frequency shows up and to see what other noise may show up and what it may look like. Past that I am pretty skeptical it could provide much information for a model. The primary reason being that FT decomposes the signal into a sum of arbitrary frequencies. We don’t have arbitrary frequencies. We know they are approximately 80 years.

What we would theoretically be going after is getting a global model that tracks the intensity of some energy/emotion metric. We could fit a model of a linear combination of the constituent identity groups’ generational cycles to some global graph so as to minimize the mean square error. Allow the amplitudes to vary and allow the frequencies to vary within some constrained distance from the 80 year mean.

If we think of this similar to WWI where small conflicts spiral into something greater what comes to mind is something similar to a domino effect (http://forleadership.org/wp-content/upl ... -Thing.pdf ch.2) which would be modeled with a geometric progression. “The Master Algorithm” notes how prevalent “S curves” or logistic style curves are in nature (the first part of the function looks like an exponential or geometric style curve up close) and notes that they are common due to phase/state changes in nature. There also seems to be some natural connection to networks with this behavior (essentially what is used in neural networks). Intuitively this makes sense as it describes the adage of how you go bankrupt “slowly at first, then all at once”. It also sounds similar to accounts of how the 2008 financial crisis progressed from what I know of it.

Hope this helps. Will continue to think about this problem.

Heisenberg
Posts: 71
Joined: Mon Feb 19, 2018 2:52 pm

Re: Heisenberg: Surf the Wave-Particle (Confidence Tradeoff)

Post by Heisenberg »

Has any approach been attempted that plots specific "energy" events spatially on a map over time?

Think weather forecasting heat maps.

Heisenberg
Posts: 71
Joined: Mon Feb 19, 2018 2:52 pm

Re: Heisenberg: Surf the Wave-Particle (Confidence Tradeoff)

Post by Heisenberg »

Heisenberg wrote:Has any approach been attempted that plots specific "energy" events spatially on a map over time?

Think weather forecasting heat maps.
I say this because one common techniques in image processing is to represent an image with its FT and perform operations in the frequency space and then convert back to the image space..... Not clear on approach right now but possibly spatially represent identity groups (and sub groups?) with their respective time lines to create an GD heat map energy image and fill in details with "hot spots" where high energy emotionally charged events such as reported by John take place. Patterns seem to hit you in the face if you can visualize them. Plus there would be a large body of well defined work on what could and could not be done. Image sharpening stands out as particularly useful. Who knows potentially image restoration: "Image restoration is based on the attempt to improve the quality of an image through knowledge of the physical processes which led to its formation". physical process = GD? Image restoration techniques commonly perform operations in the frequency domain too.

Heisenberg
Posts: 71
Joined: Mon Feb 19, 2018 2:52 pm

Re: Heisenberg: Surf the Wave-Particle (Confidence Tradeoff)

Post by Heisenberg »

I'm sure the low frequency "inevitability" side of the picture is well understood and not as important. Additionally, the farther one looks back in time the less data one would have. If we are trying to filter out noise in the high frequency side of the picture it may be beneficial to talk to a high frequency trader and see what their algorithms can do. With all the money that is thrown at those systems there might be something there. Obviously you wouldn't have the data or the desire to look at milliseconds but if we expand the time window out that may provide some beneficial info.

John
Posts: 11478
Joined: Sat Sep 20, 2008 12:10 pm
Location: Cambridge, MA USA
Contact:

Re: Heisenberg: Surf the Wave-Particle (Confidence Tradeoff)

Post by John »

Heisenberg wrote: > I'm sure the low frequency "inevitability" side of the picture is
> well understood and not as important. Additionally, the farther
> one looks back in time the less data one would have. If we are
> trying to filter out noise in the high frequency side of the
> picture it may be beneficial to talk to a high frequency trader
> and see what their algorithms can do. With all the money that is
> thrown at those systems there might be something there. Obviously
> you wouldn't have the data or the desire to look at milliseconds
> but if we expand the time window out that may provide some
> beneficial info.

If you want to look at data at the millisecond level, then modify your
software so that it creates a log file into which it stores every
action it takes, on a millisecond basis. Then you can look at the log
file later at your leisure.

Heisenberg
Posts: 71
Joined: Mon Feb 19, 2018 2:52 pm

Re: Heisenberg: Surf the Wave-Particle (Confidence Tradeoff)

Post by Heisenberg »

Ideally they might have specific equations that could precisely say how the accuracy of the model increases as we look at larger and larger periods of time and how well/far it extends into the future. It seems like this would probably be approximately what we would want. Obviously this would have to be adapted if transfered to other "energy metrics" but that likely wouldn't be much more than calculating a constant multiple is my guess. Once you have that you could play with the trade off of time frames and accuracies. If you have a specific time in mind you could optimize the parameters and time window of the model such that it maximizes the probability of the model being right at that time.

Heisenberg
Posts: 71
Joined: Mon Feb 19, 2018 2:52 pm

Re: Heisenberg: Surf the Wave-Particle (Confidence Tradeoff)

Post by Heisenberg »

For clarity too, I don't know if I would refer to timing as high frequency and inevitability as low frequency. This may be my ignorance but, from my perspective by just watching the video repeatedly, it is that when our signal ("energy metric") is looked at in narrow time spans there is a wide range of possible frequencies that could describe our signal (there is no clear dominant frequency) this is represented as a very "spread out" FT whereas when we look at the signal over larger periods of time the FT is more "narrow" and the dominant frequency becomes clear. In the video 6:51-7:03 displays this. John's model is based on incredibly long periods of time and thus the dominant frequency becomes evident (80 years). We would want to concentrate on smaller time windows, obviously, and thus the dominant frequency we are observing in those time windows becomes less clear (more possible frequencies describe the signal). We would want to filter out all the noise to be able to find the most likely dominant frequency that would describe the signal and thus give us a better idea of how it will fluctuate in the future.

FishbellykanakaDude
Posts: 1313
Joined: Tue Jan 09, 2018 8:07 pm

Re: Heisenberg: Surf the Wave-Particle (Confidence Tradeoff)

Post by FishbellykanakaDude »

Heisenberg wrote:For clarity too, I don't know if I would refer to timing as high frequency and inevitability as low frequency. This may be my ignorance but, from my perspective by just watching the video repeatedly, it is that when our signal ("energy metric") is looked at in narrow time spans there is a wide range of possible frequencies that could describe our signal (there is no clear dominant frequency) this is represented as a very "spread out" FT whereas when we look at the signal over larger periods of time the FT is more "narrow" and the dominant frequency becomes clear. In the video 6:51-7:03 displays this. John's model is based on incredibly long periods of time and thus the dominant frequency becomes evident (80 years). We would want to concentrate on smaller time windows, obviously, and thus the dominant frequency we are observing in those time windows becomes less clear (more possible frequencies describe the signal). We would want to filter out all the noise to be able to find the most likely dominant frequency that would describe the signal and thus give us a better idea of how it will fluctuate in the future.
Long "normal" Lifespan = 88ish years
Influential "normal" life-period = 58ish years
Procreational "generative" life-period = 22ish years
Freakinʻ Sunspot period = 11ish years

The "ish-ness" of these cycles makes them hard to deal with AS cycles. It seems like perhaps it may be better to see a "terrain", composed of "soils and lumpy structures", that is "weathered" by those "organic cycles" that produce the highly variable "resultant cycles" that are predicted by GD.

In other words, a "landform" is created by the "GD Weather/Hydrology/Geology" that describes (as an instantaneous/moment picture) the past and probable near future (or further) of a particular location on that landform.

What are the "movers" of GD theory? Emotions. Mostly, it seems, "frustrations". That could be the "water" that seasonally gets dumped on the "terrain". What it "pushes" around might be populations or institutions that create hills (barriers) or valleys (flow-ways), that build or erode over time.

Iʻm fishing here, quite obviously. Visualizing, or better yet "tactile-izing", the "tendencies" described by GD theory would be quite useful.

There DO seem to be a LOT of dimensions to account for, though.

Heisenberg
Posts: 71
Joined: Mon Feb 19, 2018 2:52 pm

Re: Heisenberg: Surf the Wave-Particle (Confidence Tradeoff)

Post by Heisenberg »

If any visualization is possible it would need to reduce all the variables into one output via some function because you would need 2 dimensions for the spatial representation of earth. There are plenty of techniques for dimensionality reduction. PCA is the tried and true if you don't care about being able to interpret the lower dimension space it is projected to which for the output wouldn't matter. K-means, and other clustering, techniques would likely be interesting and potentially useful for dimensionality reduction. K could be the number of identity groups, a discrete representation of age groups (adolescence, young adult, midlife, elder), etc. . One could try clustering along those separate dimensions and in the final energy metric output to see how well they correspond with identity groups. For simplicity, it is best to use a map of earth that preserves euclidean distance as that is a typical assumption in many of the available techniques. One technique that has also been useful is having high level clusters and clustering within each cluster to form a "vocabulary" of the feature space (https://en.wikipedia.org/wiki/Bag-of-wo ... ter_vision). An analysis could also be done on how these clusters change with time and if there is any correspondence with the identity group concepts.

Post Reply

Who is online

Users browsing this forum: No registered users and 8 guests