Elton  Bogan

Elton Bogan

1601098200

Predicting the flow of the South Fork Payette River using an LSTM neural network

TL;DR:

I made an LSTM neural network model that uses 30+ years of weather and streamflow data to quite accurately predict what the streamflow will be tomorrow.

The problem with river forecasts

The main reason I practice data science is to apply it to real-world problems. As a kayaker, I have spent many, many hours poring over weather forecasts, hydrologic forecasts, and SNOTEL station data to make a prediction about a river’s flow. There are good places out there that make this prediction- NOAA runs prediction centers throughout each major river basin in the country, including the South Fork.

But these forecasts often fall short. In particular, I’ve noticed that the forecasts are susceptible to major rain events (flashy rivers in the Pacific Northwest are notoriously hard to predict), and the forecasts are typically only put out once or twice per day, which is often not frequent enough to react to rapidly changing mountain weather forecasts. NOAA also only gives forecasts on a select group of rivers. If you want a forecast for a smaller or more remote drainage, even if it’s gauged, you’re out of luck.

So I’m setting out to create a model that will meet or exceed NOAA’s forecasts, and build models for some drainages that are not covered by NOAA.

To start out, I’m benchmarking my model against an industry-standard model created by Upstream Tech.

The South Fork Payette is a great place to start, for several reasons:

  1. The South Fork above Lowman is undammed, so the confounding variables of reservoirs are avoided.
  2. The USGS operates a gauge on the South Fork, NOAA has weather stations and a river forecast, and there are SNOTEL sites in the basin. There is a lot of easily accessible data to start with.
  3. I used to teach kayaking on the Payette and I’ve paddled almost every section of the river system, so I know the region and its hydrology well!

#machine-learning #lstm #hydrology #idaho #neural-networks

What is GEEK

Buddha Community

Predicting the flow of the South Fork Payette River using an LSTM neural network
Elton  Bogan

Elton Bogan

1601098200

Predicting the flow of the South Fork Payette River using an LSTM neural network

TL;DR:

I made an LSTM neural network model that uses 30+ years of weather and streamflow data to quite accurately predict what the streamflow will be tomorrow.

The problem with river forecasts

The main reason I practice data science is to apply it to real-world problems. As a kayaker, I have spent many, many hours poring over weather forecasts, hydrologic forecasts, and SNOTEL station data to make a prediction about a river’s flow. There are good places out there that make this prediction- NOAA runs prediction centers throughout each major river basin in the country, including the South Fork.

But these forecasts often fall short. In particular, I’ve noticed that the forecasts are susceptible to major rain events (flashy rivers in the Pacific Northwest are notoriously hard to predict), and the forecasts are typically only put out once or twice per day, which is often not frequent enough to react to rapidly changing mountain weather forecasts. NOAA also only gives forecasts on a select group of rivers. If you want a forecast for a smaller or more remote drainage, even if it’s gauged, you’re out of luck.

So I’m setting out to create a model that will meet or exceed NOAA’s forecasts, and build models for some drainages that are not covered by NOAA.

To start out, I’m benchmarking my model against an industry-standard model created by Upstream Tech.

The South Fork Payette is a great place to start, for several reasons:

  1. The South Fork above Lowman is undammed, so the confounding variables of reservoirs are avoided.
  2. The USGS operates a gauge on the South Fork, NOAA has weather stations and a river forecast, and there are SNOTEL sites in the basin. There is a lot of easily accessible data to start with.
  3. I used to teach kayaking on the Payette and I’ve paddled almost every section of the river system, so I know the region and its hydrology well!

#machine-learning #lstm #hydrology #idaho #neural-networks

Noah  Rowe

Noah Rowe

1597632540

Machine Translation using Neural Networks

Neural machine translation (NMT) is an approach to machine translation that uses an artificial neural network to predict the likelihood of a sequence of words, typically modeling entire sentences in a single integrated model.

Before beginning with machine translation using neural networks, first, we need to see how can we represent words in a sequence as your model works on equations and numbers and it has no place for words it only understands numbers. To do this we use tokenization. Now, what is tokenization? Tokenization is used to create a word dictionary which is there in your corpus stripping the punctuation out, conducting stemming and converting them to lower case, then we assign a specific number to each of the words, called as tokens. This gives us a dictionary where each word is mapped to a specific token. After that, we apply one-hot encoding in order to prevent words with higher token values that have a higher priority or more weight. Machine translation uses encoder-decoder architecture as shown below

Encoder and decoder both use the same neural network model but play a somewhat different role. The encoder is used to encode all the word embeddings and extract context and long term dependencies which are then passed over to decoder to generate output sentence. There are different types of natural language processing models that can be used for this purpose. Now let’s start with the basic sequence model known as Recurrent Neural Networks.

Recurrent Neural Networks

The word recurrent means occurring often or repeatedly. In normal neural networks, we take an input x and feed it forward through our activation units in our hidden layers to get an output y, we do not take any input from the previous steps in the model. This is where we differ in recurrent neural networks, in rnns we not only get data from x[t] at step t but we also get information from a[t-1](activation at the previous step), we do this in order to share features learned across different positions of texts.

#attention #recurrent-neural-network #transformers #machine-translation #lstm #neural networks

Mckenzie  Osiki

Mckenzie Osiki

1623135499

No Code introduction to Neural Networks

The simple architecture explained

Neural networks have been around for a long time, being developed in the 1960s as a way to simulate neural activity for the development of artificial intelligence systems. However, since then they have developed into a useful analytical tool often used in replace of, or in conjunction with, standard statistical models such as regression or classification as they can be used to predict or more a specific output. The main difference, and advantage, in this regard is that neural networks make no initial assumptions as to the form of the relationship or distribution that underlies the data, meaning they can be more flexible and capture non-standard and non-linear relationships between input and output variables, making them incredibly valuable in todays data rich environment.

In this sense, their use has took over the past decade or so, with the fall in costs and increase in ability of general computing power, the rise of large datasets allowing these models to be trained, and the development of frameworks such as TensforFlow and Keras that have allowed people with sufficient hardware (in some cases this is no longer even an requirement through cloud computing), the correct data and an understanding of a given coding language to implement them. This article therefore seeks to be provide a no code introduction to their architecture and how they work so that their implementation and benefits can be better understood.

Firstly, the way these models work is that there is an input layer, one or more hidden layers and an output layer, each of which are connected by layers of synaptic weights¹. The input layer (X) is used to take in scaled values of the input, usually within a standardised range of 0–1. The hidden layers (Z) are then used to define the relationship between the input and output using weights and activation functions. The output layer (Y) then transforms the results from the hidden layers into the predicted values, often also scaled to be within 0–1. The synaptic weights (W) connecting these layers are used in model training to determine the weights assigned to each input and prediction in order to get the best model fit. Visually, this is represented as:

#machine-learning #python #neural-networks #tensorflow #neural-network-algorithm #no code introduction to neural networks

The Recurrent Neural Network (RNNs)

A recurrent neural network (RNN) is an input node (hidden layer) that feeds sigmoid activation. The way an RNN does this is to take the output of one neuron and return it as input to another neuron or feed the input of the current time step to the output of earlier time steps. Here you feed the input from the previous times step by step into the input of the current times and vice versa.

Image for post

This can be used in a variety of ways, such as through learning gates with known variations or a combination of sigmoid activation and a number of other types of neural networks.

Some of the applications for RNNs include predicting energy demand, predicting stock prices, and predicting human behavior. RNNs are modeled over time — based and sequence-based data, but they are also useful in a variety of other applications.

A recurrent neural network is an artificial neural network used for deep learning, machine learning, and other forms of artificial intelligence (AI). They have a number of attributes that make them useful for tasks where data needs to be processed sequentially.

To get a little more technical, recurring neural networks are designed to learn a sequence of data by traversing a hidden state from one step of the sequence to the next, combined with the input, and routing it back and forth between the inputs. RNN are neural networks that are designed for the effective handling of sequential data but are also useful for non-sequential data.

These types of data include text documents that can be seen as a sequence of words or audio files in which you can see a sequence of sound frequencies and times. The more information about the output layer is available, the faster it can be read and sequenced, and the better its performance.

#recurrent-neural-network #lstm #rnn #artificial-intelligence #neural network

Multiple Stock Prediction Using Deep Learning Network

Stock prediction is a difficult task because the data generated is enormous and is highly non-linear. Modeling such dynamical data require effective modeling technique which can analyze the hidden patterns and underlying dynamics. Neural network which is deep learning algorithms are capable of identifying and exploiting the interactions and patterns existing in a data through a self learning process.We shall focus on a simple prediction model using long-short term memory architecture involving multiple time-series. We shall keep the whole exercise a simple process for illustration purpose and better understanding.Let us consider four different series S&P 500 (^GSPC), Dow Jones Industrial Average (^DJI), NASDAQ Composite (^IXIC) and Russell 2000 (^RUT) which is Chicago options. Our data set if from 1990 to till date.

stock = ['^RUT', '^GSPC', '^DJI', '^IXIC' ]
start = pd.to_datetime('1990-01-03')
df = web.DataReader(stock, data_source = 'yahoo', start = start )

Image for post

Let us fit all the closing prices of the series in a neural network architecture and shall predict one series (NASDAQ composite price) from there.Train-test splitWe split into train-test and visualize the output.

data = df [['Close']]
data = data.reset_index()

training_data = data[data['Date'] < pd.to_datetime('2016-12-31')].copy()
test_data = data[data['Date'] >= pd.to_datetime("2016-12-31")].copy()
training_data = training_data.set_index('Date')
test_data = test_data.set_index('Date')
plt.figure(figsize=(14,4))
plt.plot(training_data.Close['^RUT'])
plt.plot(test_data.Close['^RUT'])
plt.ylabel("Price")
plt.xlabel("Date")
plt.legend(["Training Set", "Test Set"])
plt.title("Chicago Options Close Price")
plt.show()

Image for post](https://miro.medium.com/max/1133/1*VFcqWLy2ScRZSTWCRoU7mw.png)

Likewise, we can draw other plots in the series too.

#neural-networks #predictive-analytics #lstm