Before Predicting Gold: Exploring ETF’s Tracking Error

“ As humans, we are attracted to the colorful leaves and sometimes pay little attention to the roots, the stem, and the branches that are essential to the tree’s life. This is comparable to doing a financial model where a novice Quant rushes to the modeling process without spending sufficient time exploring the intricate details of the data”.

LBMA PM Fix & GLD ETF from Notebook

In this article, I will share my personal experience in doing a research project for Global Precious Metals. My firm is a niche player in the precious metals market, offering global end-to-end service from sales and trading to storing and delivering precious metals to prospective clients. Their goal was to ‘bring some quantitative practice to old school physical work’. I was to take part in this revolution and help the firm realize a systematic way to allocate money to gold. The firm might not be familiar with deriving the mathematical cost function of a regression model, but they know the mechanics of the gold industry. I was always astonished by the heavy flow of information and findings revolving this polarized asset, which kindled my exploration from the very first day.I told myself “forget about modeling, it’s time to learn about gold”.There is a plethora of benefits and caveats of using gold ETF as an alternative to gold spot prices for modeling. I will also allude to the notion of fund tracking errors and underlying factors that caused it. Tracking error exists in every fund that tracks a benchmark. Because it exists, one has to question whether or not the gold ETF would represent the genuine value of gold. It introduces caution to not only gold modelers but also investors. This article will not go through the deep mechanism of tracking errors, but it will guide you through my resolve to it.The article will address the following considerations before developing a model for gold:

• Common gold assets to based our model on and why ETFs are a feasible choice_Calculating/Visualizing tracking errors for different gold ETFs_Verifying factors that affect the tracking error of gold ETFsChoosing a gold ETF based on tracking error

A Little Detail on Gold Choices

So we want to create an investment model for gold. The first question is which gold asset do we model with. Some of the asset choices are:

1. Physical gold: The London Bullion OTC market (LBMA) offers investors gold as a tangible asset. Holding gold this way can sustain long term wealth without any counter party risks. Even though LBMA is based in London, the market can be traded from across the globe. The LBMA website holds some useful information on setting prices and trading terms.Exchange futures contracts: Futures contract traders are composed of mainly hedgers and speculators. Hedgers would buy/sell the present contract price of gold they wish to hold (to be delivered in future) and hedge the risk of the price rising/falling before the expiry date. Speculators conduct trades in order to profit from their speculation of price movement, as their goal is not to hold physical gold itself.Gold Tracking Exchange Traded Fund (ETF): These type of ETFs track the returns of spot gold prices by holding physical gold and futures contracts. Investors would invest in the shares of the ETF. Futures speculators and ETF investors are similar because they trade gold without holding gold. There are other gold ETFs that does not gain exposure to the commodity directly, instead investing in companies that specializes in gold.

So which Gold to model?

Assume your objective is to invest and actually store the gold. Allocation to gold will be re-balanced weekly. We decided to use an ML model to predict the direction of one week returns using daily observations.

Using futures prices are out of the question because we are not hedging nor speculating on the futures price. We can base the model on the LBMA spot price, but this method conveys some issues when constructing our feature set.

#linear-regression #gold #error-tracking #data analysis

What is GEEK

Buddha Community

Machine Learning Tutorial: Step By Step for Beginners

In this Machine Learning article, we learn about Machine Learning Tutorial: step by step for beginners. This Machine Learning tutorial provides both intermediate and basics of machine learning. It is designed for students and working professionals who are complete beginners. At the end of this tutorial, you will be able to make machine learning models that can perform complex tasks such as predicting the price of a house or recognizing the species of an Iris from the dimensions of its petal and sepal lengths. If you are not a complete beginner and are a bit familiar with Machine Learning, I would suggest starting with subtopic eight i.e, Types of Machine Learning.

Before we deep dive further, if you are keen to explore a course in Artificial Intelligence & Machine Learning do check out our Artificial Intelligence Courses available at Great Learning. Anyone could expect an average Salary Hike of 48% from this course. Participate in Great Learning’s career accelerate programs and placement drives and get hired by our pool of 500+ Hiring companies through our programs.

Before jumping into the tutorial, you should be familiar with Pandas and NumPy. This is important to understand the implementation part. There are no prerequisites for understanding the theory. Here are the subtopics that we are going to discuss in this tutorial:

What is Machine Learning?

Arthur Samuel coined the term Machine Learning in the year 1959. He was a pioneer in Artificial Intelligence and computer gaming, and defined Machine Learning as “Field of study that gives computers the capability to learn without being explicitly programmed”.

In simple terms, Machine Learning is an application of Artificial Intelligence (AI) which enables a program(software) to learn from the experiences and improve their self at a task without being explicitly programmed. For example, how would you write a program that can identify fruits based on their various properties, such as colour, shape, size or any other property?

One approach is to hardcode everything, make some rules and use them to identify the fruits. This may seem the only way and work but one can never make perfect rules that apply on all cases. This problem can be easily solved using machine learning without any rules which makes it more robust and practical. You will see how we will use machine learning to do this task in the coming sections.

Thus, we can say that Machine Learning is the study of making machines more human-like in their behaviour and decision making by giving them the ability to learn with minimum human intervention, i.e., no explicit programming. Now the question arises, how can a program attain any experience and from where does it learn? The answer is data. Data is also called the fuel for Machine Learning and we can safely say that there is no machine learning without data.

You may be wondering that the term Machine Learning has been introduced in 1959 which is a long way back, then why haven’t there been any mention of it till recent years? You may want to note that Machine Learning needs a huge computational power, a lot of data and devices which are capable of storing such vast data. We have only recently reached a point where we now have all these requirements and can practice Machine Learning.

How is it different from traditional programming?

Are you wondering how is Machine Learning different from traditional programming? Well, in traditional programming, we would feed the input data and a well written and tested program into a machine to generate output. When it comes to machine learning, input data along with the output associated with the data is fed into the machine during the learning phase, and it works out a program for itself.

Why do we need Machine Learning?

Machine Learning today has all the attention it needs. Machine Learning can automate many tasks, especially the ones that only humans can perform with their innate intelligence. Replicating this intelligence to machines can be achieved only with the help of machine learning. 

With the help of Machine Learning, businesses can automate routine tasks. It also helps in automating and quickly create models for data analysis. Various industries depend on vast quantities of data to optimize their operations and make intelligent decisions. Machine Learning helps in creating models that can process and analyze large amounts of complex data to deliver accurate results. These models are precise and scalable and function with less turnaround time. By building such precise Machine Learning models, businesses can leverage profitable opportunities and avoid unknown risks.

Image recognition, text generation, and many other use-cases are finding applications in the real world. This is increasing the scope for machine learning experts to shine as a sought after professionals. 

How Does Machine Learning Work?

A machine learning model learns from the historical data fed to it and then builds prediction algorithms to predict the output for the new set of data the comes in as input to the system. The accuracy of these models would depend on the quality and amount of input data. A large amount of data will help build a better model which predicts the output more accurately.

Suppose we have a complex problem at hand that requires to perform some predictions. Now, instead of writing a code, this problem could be solved by feeding the given data to generic machine learning algorithms. With the help of these algorithms, the machine will develop logic and predict the output. Machine learning has transformed the way we approach business and social problems. Below is a diagram that briefly explains the working of a machine learning model/ algorithm. our way of thinking about the problem.

History of Machine Learning

Nowadays, we can see some amazing applications of ML such as in self-driving cars, Natural Language Processing and many more. But Machine learning has been here for over 70 years now. It all started in 1943, when neurophysiologist Warren McCulloch and mathematician Walter Pitts wrote a paper about neurons, and how they work. They decided to create a model of this using an electrical circuit, and therefore, the neural network was born.

In 1950, Alan Turing created the “Turing Test” to determine if a computer has real intelligence. To pass the test, a computer must be able to fool a human into believing it is also human. In 1952, Arthur Samuel wrote the first computer learning program. The program was the game of checkers, and the IBM computer improved at the game the more it played, studying which moves made up winning strategies and incorporating those moves into its program.

Just after a few years, in 1957, Frank Rosenblatt designed the first neural network for computers (the perceptron), which simulates the thought processes of the human brain. Later, in 1967, the “nearest neighbor” algorithm was written, allowing computers to begin using very basic pattern recognition. This could be used to map a route for travelling salesmen, starting at a random city but ensuring they visit all cities during a short tour.

But we can say that in the 1990s we saw a big change. Now work on machine learning shifted from a knowledge-driven approach to a data-driven approach.  Scientists began to create programs for computers to analyze large amounts of data and draw conclusions or “learn” from the results.

In 1997, IBM’s Deep Blue became the first computer chess-playing system to beat a reigning world chess champion. Deep Blue used the computing power in the 1990s to perform large-scale searches of potential moves and select the best move. Just a decade before this, in 2006, Geoffrey Hinton created the term “deep learning” to explain new algorithms that help computers distinguish objects and text in images and videos.

Machine Learning at Present

The year 2012 saw the publication of an influential research paper by Alex Krizhevsky, Geoffrey Hinton, and Ilya Sutskever, describing a model that can dramatically reduce the error rate in image recognition systems. Meanwhile, Google’s X Lab developed a machine learning algorithm capable of autonomously browsing YouTube videos to identify the videos that contain cats. In 2016 AlphaGo (created by researchers at Google DeepMind to play the ancient Chinese game of Go) won four out of five matches against Lee Sedol, who has been the world’s top Go player for over a decade.

And now in 2020, OpenAI released GPT-3 which is the most powerful language model ever. It can write creative fiction, generate functioning code, compose thoughtful business memos and much more. Its possible use cases are limited only by our imaginations.

Features of Machine Learning

1. Automation: Nowadays in your Gmail account, there is a spam folder that contains all the spam emails. You might be wondering how does Gmail know that all these emails are spam? This is the work of Machine Learning. It recognizes the spam emails and thus, it is easy to automate this process. The ability to automate repetitive tasks is one of the biggest characteristics of machine learning. A huge number of organizations are already using machine learning-powered paperwork and email automation. In the financial sector, for example, a huge number of repetitive, data-heavy and predictable tasks are needed to be performed. Because of this, this sector uses different types of machine learning solutions to a great extent.

2. Improved customer experience: For any business, one of the most crucial ways to drive engagement, promote brand loyalty and establish long-lasting customer relationships is by providing a customized experience and providing better services. Machine Learning helps us to achieve both of them. Have you ever noticed that whenever you open any shopping site or see any ads on the internet, they are mostly about something that you recently searched for? This is because machine learning has enabled us to make amazing recommendation systems that are accurate. They help us customize the user experience. Now coming to the service, most of the companies nowadays have a chatting bot with them that are available 24×7. An example of this is Eva from AirAsia airlines. These bots provide intelligent answers and sometimes you might even not notice that you are having a conversation with a bot. These bots use Machine Learning, which helps them to provide a good user experience.

3. Automated data visualization: In the past, we have seen a huge amount of data being generated by companies and individuals. Take an example of companies like Google, Twitter, Facebook. How much data are they generating per day? We can use this data and visualize the notable relationships, thus giving businesses the ability to make better decisions that can actually benefit both companies as well as customers. With the help of user-friendly automated data visualization platforms such as AutoViz, businesses can obtain a wealth of new insights in an effort to increase productivity in their processes.

4. Business intelligence: Machine learning characteristics, when merged with big data analytics can help companies to find solutions to the problems that can help the businesses to grow and generate more profit. From retail to financial services to healthcare, and many more, ML has already become one of the most effective technologies to boost business operations.

Python provides flexibility in choosing between object-oriented programming or scripting. There is also no need to recompile the code; developers can implement any changes and instantly see the results. You can use Python along with other languages to achieve the desired functionality and results.

Python is a versatile programming language and can run on any platform including Windows, MacOS, Linux, Unix, and others. While migrating from one platform to another, the code needs some minor adaptations and changes, and it is ready to work on the new platform. To build strong foundation and cover basic concepts you can enroll in a python machine learning course that will help you power ahead your career.

Here is a summary of the benefits of using Python for Machine Learning problems:

Types of Machine Learning

Machine learning has been broadly categorized into three categories

1. Supervised Learning
2. Unsupervised Learning
3. Reinforcement Learning

What is Supervised Learning?

Let us start with an easy example, say you are teaching a kid to differentiate dogs from cats. How would you do it? 

You may show him/her a dog and say “here is a dog” and when you encounter a cat you would point it out as a cat. When you show the kid enough dogs and cats, he may learn to differentiate between them. If he is trained well, he may be able to recognize different breeds of dogs which he hasn’t even seen. 

Similarly, in Supervised Learning, we have two sets of variables. One is called the target variable, or labels (the variable we want to predict) and features(variables that help us to predict target variables). We show the program(model) the features and the label associated with these features and then the program is able to find the underlying pattern in the data. Take this example of the dataset where we want to predict the price of the house given its size. The price which is a target variable depends upon the size which is a feature.

Number of rooms	Price
1	$100
3	$300
5	$500

In a real dataset, we will have a lot more rows and more than one features like size, location, number of floors and many more.

Thus, we can say that the supervised learning model has a set of input variables (x), and an output variable (y). An algorithm identifies the mapping function between the input and output variables. The relationship is y = f(x).

The learning is monitored or supervised in the sense that we already know the output and the algorithm are corrected each time to optimize its results. The algorithm is trained over the data set and amended until it achieves an acceptable level of performance.

We can group the supervised learning problems as:

Regression problems – Used to predict future values and the model is trained with the historical data. E.g., Predicting the future price of a house.

Classification problems – Various labels train the algorithm to identify items within a specific category. E.g., Dog or cat( as mentioned in the above example), Apple or an orange, Beer or wine or water.

What is Unsupervised Learning?

This approach is the one where we have no target variables, and we have only the input variable(features) at hand. The algorithm learns by itself and discovers an impressive structure in the data. 

The goal is to decipher the underlying distribution in the data to gain more knowledge about the data. 

We can group the unsupervised learning problems as:

Clustering: This means bundling the input variables with the same characteristics together. E.g., grouping users based on search history

Association: Here, we discover the rules that govern meaningful associations among the data set. E.g., People who watch ‘X’ will also watch ‘Y’.

What is Reinforcement Learning?

In this approach, machine learning models are trained to make a series of decisions based on the rewards and feedback they receive for their actions. The machine learns to achieve a goal in complex and uncertain situations and is rewarded each time it achieves it during the learning period. 

Reinforcement learning is different from supervised learning in the sense that there is no answer available, so the reinforcement agent decides the steps to perform a task. The machine learns from its own experiences when there is no training data set present.

In this tutorial, we are going to mainly focus on Supervised Learning and Unsupervised learning as these are quite easy to understand and implement.

Machine learning Algorithms

This may be the most time-consuming and difficult process in your journey of Machine Learning. There are many algorithms in Machine Learning and you don’t need to know them all in order to get started. But I would suggest, once you start practising Machine Learning, start learning about the most popular algorithms out there such as:

• Linear Regression
• Logistic Regression
• Decision Tree
• SVM
• Naive Bayes
• K-nearest neighbour
• K-Means
• Random Forest
• Gradient Boosting algorithms
  • GBM
  • XGBoost
  • LightGBM
  • CatBoost

Here, I am going to give a brief overview of one of the simplest algorithms in Machine learning, the K-nearest neighbor Algorithm (which is a Supervised learning algorithm) and show how we can use it for Regression as well as for classification. I would highly recommend checking the Linear Regression and Logistic Regression as we are going to implement them and compare the results with KNN(K-nearest neighbor) algorithm in the implementation part.

You may want to note that there are usually separate algorithms for regression problems and classification problems. But by modifying an algorithm, we can use it for both classifications as well as regression as you will see below

K-Nearest Neighbor Algorithm

KNN belongs to a group of lazy learners. As opposed to eager learners such as logistic regression, SVM, neural nets, lazy learners just store the training data in memory. During the training phase, KNN arranges the data (sort of indexing process) in order to find the closest neighbours efficiently during the inference phase. Otherwise, it would have to compare each new case during inference with the whole dataset making it quite inefficient.

So if you are wondering what is a training phase, eager learners and lazy learners, for now just remember that training phase is when an algorithm learns from the data provided to it. For example, if you have gone through the Linear Regression algorithm linked above, during the training phase the algorithm tries to find the best fit line which is a process that includes a lot of computations and hence takes a lot of time and this type of algorithm is called eager learners. On the other hand, lazy learners are just like KNN which do not involve many computations and hence train faster.

K-NN for Classification Problem

Now let us see how we can use K-NN for classification. Here a hypothetical dataset which tries to predict if a person is male or female (labels) on the base of the height and weight (features).

Height(cm) -feature	Weight(kg) -feature.	Gender(label)
187	80	Male
165	50	Female
199	99	Male
145	70	Female
180	87	Male
178	65	Female
187	60	Male

Now let us plot these points:

Now we have a new point that we want to classify, given that its height is 190 cm and weight is 100 Kg. Here is how K-NN will classify this point:

1. Select the value of K, which the user selects which he thinks will be best after analysing the data.
2. Measure the distance of new points from its nearest K number of points. There are various methods for calculating this distance, of which the most commonly known methods are – Euclidian, Manhattan (for continuous data points i.e regression problems) and Hamming distance (for categorical i.e for classification problems).
3. Identify the class of the points that are more closer to the new point and label the new point accordingly. So if the majority of points closer to our new point belong to a certain “a” class than our new point is predicted to be from class “a”.

Now let us apply this algorithm to our own dataset. Let us first plot the new data point.

Now let us take k=3 i.e, we will see the three closest points to the new point:

Therefore, it is classified as Male:

Now let us take the value of k=5 and see what happens:

As we can see four of the points closest to our new data point are males and just one point is female, so we go with the majority and classify it as Male again. You must always select the value of K as an odd number when doing classification.

K-NN for a Regression problem

We have seen how we can use K-NN for classification. Now, let us see what changes are made to use it for regression. The algorithm is almost the same there is just one difference. In Classification, we checked for the majority of all nearest points. Here, we are going to take the average of all the nearest points and take that as predicted value. Let us again take the same example but here we have to predict the weight(label) of a person given his height(features).

Height(cm) -feature	Weight(kg) -label
187	80
165	50
199	99
145	70
180	87
178	65
187	60

Now we have new data point with a height of 160cm, we will predict its weight by taking the values of K as 1,2 and 4.

When K=1: The closest point to 160cm in our data is 165cm which has a weight of 50, so we conclude that the predicted weight is 50 itself.

When K=2: The two closest points are 165 and 145 which have weights equal to 50 and 70 respectively. Taking average we say that the predicted weight is (50+70)/2=60.

When K=4: Repeating the same process, now we take 4 closest points instead and hence we get 70.6 as predicted weight.

You might be thinking that this is really simple and there is nothing so special about Machine learning, it is just basic Mathematics. But remember this is the simplest algorithm and you will see much more complex algorithms once you move ahead in this journey.

At this stage, you must have a vague idea of how machine learning works, don’t worry if you are still confused. Also if you want to go a bit deep now, here is an excellent article – Gradient Descent in Machine Learning, which discusses how we use an optimization technique called as gradient descent to find a best-fit line in linear regression.

How To Choose Machine Learning Algorithm?

There are plenty of machine learning algorithms and it could be a tough task to decide which algorithm to choose for a specific application. The choice of the algorithm will depend on the objective of the problem you are trying to solve.

Let us take an example of a task to predict the type of fruit among three varieties, i.e., apple, banana, and orange. The predictions are based on the colour of the fruit. The picture depicts the results of ten different algorithms. The picture on the top left is the dataset. The data is classified into three categories: red, light blue and dark blue. There are some groupings. For instance, from the second image, everything in the upper left belongs to the red category, in the middle part, there is a mixture of uncertainty and light blue while the bottom corresponds to the dark category. The other images show different algorithms and how they try to classified the data.

Steps in Machine Learning

I wish Machine learning was just applying algorithms on your data and get the predicted values but it is not that simple. There are several steps in Machine Learning which are must for each project.

1. Gathering Data: This is perhaps the most important and time-consuming process. In this step, we need to collect data that can help us to solve our problem. For example, if you want to predict the prices of the houses, we need an appropriate dataset that contains all the information about past house sales and then form a tabular structure. We are going to solve a similar problem in the implementation part.
2. Preparing that data: Once we have the data, we need to bring it in proper format and preprocess it. There are various steps involved in pre-processing such as data cleaning, for example, if your dataset has some empty values or abnormal values(e.g, a string instead of a number) how are you going to deal with it? There are various ways in which we can but one simple way is to just drop the rows that have empty values. Also sometimes in the dataset, we might have columns that have no impact on our results such as id’s, we remove those columns as well. We usually use Data Visualization to visualize our data through graphs and diagrams and after analyzing the graphs, we decide which features are important. Data preprocessing is a vast topic and I would suggest checking out this article to know more about it.
3. Choosing a model: Now our data is ready is to be fed into a Machine Learning algorithm. In case you are wondering what is a Model? Often “machine learning algorithm” is used interchangeably with “machine learning model.” A model is the output of a machine learning algorithm run on data. In simple terms when we implement the algorithm on all our data, we get an output which contains all the rules, numbers, and any other algorithm-specific data structures required to make predictions. For example, after implementing Linear Regression on our data we get an equation of the best fit line and this equation is termed as a model. The next step is usually training the model incase we don’t want to tune hyperparameters and select the default ones.
4. Hyperparameter Tuning: Hyperparameters are crucial as they control the overall behavior of a machine learning model. The ultimate goal is to find an optimal combination of hyperparameters that gives us the best results. But what are these hyper-parameters? Remember the variable K in our K-NN algorithm. We got different results when we set different values of K. The best value for K is not predefined and is different for different datasets. There is no method to know the best value for K, but you can try different values and check for which value do we get the best results. Here K is a hyperparameter and each algorithm has its own hyperparameters and we need to tune their values to get the best results. To get more information about it, check out this article – Hyperparameter Tuning Explained.
5. Evaluation: You may be wondering, how can you know if the model is performing good or bad. What better way than testing the model on some data. This data is known as testing data and it must not be a subset of the data (training data) on which we trained the algorithm. The objective of training the model is not for it to learn all the values in the training dataset but to identify the underlying pattern in data and based on that make predictions on data it has never seen before. There are various evaluation methods such as K-fold cross-validation and many more. We are going to discuss this step in detail in the coming section.
6. Prediction: Now that our model has performed well on the testing set as well, we can use it in real-world and hope it is going to perform well on real-world data.

Evaluation of Machine learning Model

For evaluating the model, we hold out a portion of data called test data and do not use this data to train the model. Later, we use test data to evaluate various metrics.

The results of predictive models can be viewed in various forms such as by using confusion matrix, root-mean-squared error(RMSE), AUC-ROC etc.

TP (True Positive) is the number of values predicted to be positive by the algorithm and was actually positive in the dataset. TN represents the number of values that are expected to not belong to the positive class and actually do not belong to it. FP depicts the number of instances misclassified as belonging to the positive class thus is actually part of the negative class. FN shows the number of instances classified as the negative class but should belong to the positive class. 

Now in Regression problem, we usually use RMSE as evaluation metrics. In this evaluation technique, we use the error term.

Let’s say you feed a model some input X and the model predicts 10, but the actual value is 5. This difference between your prediction (10) and the actual observation (5) is the error term: (f_prediction – i_actual). The formula to calculate RMSE is given by:

Where N is a total number of samples for which we are calculating RMSE.

In a good model, the RMSE should be as low as possible and there should not be much difference between RMSE calculated over training data and RMSE calculated over the testing set. 

Python for Machine Learning

Although there are many languages that can be used for machine learning, according to me, Python is hands down the best programming language for Machine Learning applications. This is due to the various benefits mentioned in the section below. Other programming languages that could to use for Machine Learning Applications are R, C++, JavaScript, Java, C#, Julia, Shell, TypeScript, and Scala. R is also a really good language to get started with machine learning.

Python is famous for its readability and relatively lower complexity as compared to other programming languages. Machine Learning applications involve complex concepts like calculus and linear algebra which take a lot of effort and time to implement. Python helps in reducing this burden with quick implementation for the Machine Learning engineer to validate an idea. You can check out the Python Tutorial to get a basic understanding of the language. Another benefit of using Python in Machine Learning is the pre-built libraries. There are different packages for a different type of applications, as mentioned below:

1. Numpy, OpenCV, and Scikit are used when working with images
2. NLTK along with Numpy and Scikit again when working with text
3. Librosa for audio applications
4. Matplotlib, Seaborn, and Scikit for data representation
5. TensorFlow and Pytorch for Deep Learning applications
6. Scipy for Scientific Computing
7. Django for integrating web applications
8. Pandas for high-level data structures and analysis

Implementation of algorithms in Machine Learning with Python

Before moving on to the implementation of machine learning with Python part, you need to download some important software and libraries. Anaconda is an open-source distribution that makes it easy to perform Python/R data science and machine learning on a single machine. It contains all most all the libraries that are needed by us. In this tutorial, we are mostly going to use the scikit-learn library which is a free software machine learning library for the Python programming language.

Now, we are going to implement all that we learnt till now. We will solve a Regression problem and then a Classification problem using the seven steps mentioned above.

Implementation of a Regression problem

We have a problem of predicting the prices of the house given some features such as size, number of rooms and many more. So let us get started:

1. Gathering data: We don’t need to manually collect the data for past sales of houses. Luckily there are some good people who do it for us and make these datasets available for us to use. Also let me mention not all datasets are free but for you to practice, you will find most of the datasets free to use on the internet.

The dataset we are using is called the Boston Housing dataset. Each record in the database describes a Boston suburb or town. The data was drawn from the Boston Standard Metropolitan Statistical Area (SMSA) in 1970. The attributes are defined as follows (taken from the UCI Machine Learning Repository).

1. CRIM: per capita crime rate by town
2. ZN: proportion of residential land zoned for lots over 25,000 sq.ft.
3. INDUS: proportion of non-retail business acres per town
4. CHAS: Charles River dummy variable (= 1 if tract bounds river; 0 otherwise)
5. NOX: nitric oxides concentration (parts per 10 million)
6. RM: average number of rooms per dwelling
7. AGE: the proportion of owner-occupied units built prior to 1940
8. DIS: weighted distances to five Boston employment centers
9. RAD: index of accessibility to radial highways
10. TAX: full-value property-tax rate per $10,000
11. PTRATIO: pupil-teacher ratio by town 
12. B: 1000(Bk−0.63)2 where Bk is the proportion of blacks by town 
13. LSTAT: % lower status of the population
14. MEDV: Median value of owner-occupied homes in $1000s

Here is a link to download this dataset.

Now after opening the file you can see the data about House sales. This dataset is not in a proper tabular form, in fact, there are no column names and each value is separated by spaces. We are going to use Pandas to put it in proper tabular form. We will provide it with a list containing column names and also use delimiter as ‘\s+’ which means that after encounterings a single or multiple spaces, it can differentiate every single entry.

We are going to import all the necessary libraries such as Pandas and NumPy. Next, we will import the data file which is in CSV format into a pandas DataFrame.

import numpy as np
import pandas as pd
column_names = ['CRIM', 'ZN', 'INDUS', 'CHAS', 'NOX', 'RM', 'AGE', 'DIS', 'RAD', 'TAX','PTRATIO', 'B', 'LSTAT', 'MEDV']
bos1 = pd.read_csv('housing.csv', delimiter=r"\s+", names=column_names)

2. Preprocess Data: The next step is to pre-process the data. Now for this dataset, we can see that there are no NaN (missing) values and also all the data is in numbers rather than strings so we won’t face any errors when training the model. So let us just divide our data into training data and testing data such that 70% of data is training data and the rest is testing data. We could also scale our data to make the predictions much accurate but for now, let us keep it simple.

bos1.isna().sum()

from sklearn.model_selection import train_test_split
X=np.array(bos1.iloc[:,0:13])
Y=np.array(bos1["MEDV"])
#testing data size is of 30% of entire data
x_train, x_test, y_train, y_test =train_test_split(X,Y, test_size = 0.30, random_state =5)

3. Choose a Model: For this particular problem, we are going to use two algorithms of supervised learning that can solve regression problems and later compare their results. One algorithm is K-NN (K-nearest Neighbor) which is explained above and the other is Linear Regression. I would highly recommend to check it out in case you haven’t already.

from sklearn.linear_model import LinearRegression
from sklearn.neighbors import KNeighborsRegressor
#load our first model 
lr = LinearRegression()
#train the model on training data
lr.fit(x_train,y_train)
#predict the testing data so that we can later evaluate the model
pred_lr = lr.predict(x_test)
#load the second model
Nn=KNeighborsRegressor(3)
Nn.fit(x_train,y_train)
pred_Nn = Nn.predict(x_test)

4. Hyperparameter Tuning: Since this is a beginners tutorial, here, I am only going to turn the value ok K in the K-NN model. I will just use a for loop and check results of k ranging from 1 to 50. K-NN is extremely fast on small dataset like ours so it won’t take any time. There are much more advanced methods of doing this which you can find linked in the steps of Machine Learning section above.

import sklearn
for i in range(1,50):
    model=KNeighborsRegressor(i)
    model.fit(x_train,y_train)
    pred_y = model.predict(x_test)
    mse = sklearn.metrics.mean_squared_error(y_test, pred_y,squared=False)
    print("{} error for k = {}".format(mse,i))

Output:

From the output, we can see that error is least for k=3, so that should justify why I put the value of K=3 while training the model

5. Evaluating the model: For evaluating the model we are going to use the mean_squared_error() method from the scikit-learn library. Remember to set the parameter ‘squared’ as False, to get the RMSE error.

#error for linear regression
mse_lr= sklearn.metrics.mean_squared_error(y_test, pred_lr,squared=False)
print("error for Linear Regression = {}".format(mse_lr))
#error for linear regression
mse_Nn= sklearn.metrics.mean_squared_error(y_test, pred_Nn,squared=False)
print("error for K-NN = {}".format(mse_Nn))

Now from the results, we can conclude that Linear Regression performs better than K-NN for this particular dataset. But It is not necessary that Linear Regression would always perform better than K-NN as it completely depends upon the data that we are working with.

6. Prediction: Now we can use the models to predict the prices of the houses using the predict function as we did above. Make sure when predicting the prices that we are given all the features that were present when training the model.

Here is the whole script:

import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.neighbors import KNeighborsRegressor
column_names = ['CRIM', 'ZN', 'INDUS', 'CHAS', 'NOX', 'RM', 'AGE', 'DIS', 'RAD', 'TAX', 'PTRATIO', 'B', 'LSTAT', 'MEDV']
bos1 = pd.read_csv('housing.csv', delimiter=r"\s+", names=column_names)
X=np.array(bos1.iloc[:,0:13])
Y=np.array(bos1["MEDV"])
#testing data size is of 30% of entire data
x_train, x_test, y_train, y_test =train_test_split(X,Y, test_size = 0.30, random_state =54)
#load our first model 
lr = LinearRegression()
#train the model on training data
lr.fit(x_train,y_train)
#predict the testing data so that we can later evaluate the model
pred_lr = lr.predict(x_test)
#load the second model
Nn=KNeighborsRegressor(12)
Nn.fit(x_train,y_train)
pred_Nn = Nn.predict(x_test)
#error for linear regression
mse_lr= sklearn.metrics.mean_squared_error(y_test, pred_lr,squared=False)
print("error for Linear Regression = {}".format(mse_lr))
#error for linear regression
mse_Nn= sklearn.metrics.mean_squared_error(y_test, pred_Nn,squared=False)
print("error for K-NN = {}".format(mse_Nn))

Implementation of a Classification problem

In this section, we will solve the population classification problem known as Iris Classification problem. The Iris dataset was used in R.A. Fisher’s classic 1936 paper, The Use of Multiple Measurements in Taxonomic Problems, and can also be found on the UCI Machine Learning Repository.

It includes three iris species with 50 samples each as well as some properties about each flower. One flower species is linearly separable from the other two, but the other two are not linearly separable from each other. The columns in this dataset are:

Different species of iris

• SepalLengthCm
• SepalWidthCm
• PetalLengthCm
• PetalWidthCm
• Species

We don’t need to download this dataset as scikit-learn library already contains this dataset and we can simply import it from there. So let us start coding this up:

from sklearn.datasets import load_iris
iris = load_iris()
X=iris.data
Y=iris.target
print(X)
print(Y)

As we can see, the features are in a list containing four items which are the features and at the bottom, we got a list containing labels which have been transformed into numbers as the model cannot understand names that are strings, so we encode each name as a number. This has already done by the scikit learn developers.

from sklearn.model_selection import train_test_split
#testing data size is of 30% of entire data
x_train, x_test, y_train, y_test =train_test_split(X,Y, test_size = 0.3, random_state =5)
from sklearn.linear_model import LogisticRegression
from sklearn.neighbors import KNeighborsClassifier
#fitting our model to train and test
Nn = KNeighborsClassifier(8)
Nn.fit(x_train,y_train)
#the score() method calculates the accuracy of model.
print("Accuracy for K-NN is ",Nn.score(x_test,y_test))
Lr = LogisticRegression()
Lr.fit(x_train,y_train)
print("Accuracy for Logistic Regression is ",Lr.score(x_test,y_test))

Advantages of Machine Learning

1. Easily identifies trends and patterns

Machine Learning can review large volumes of data and discover specific trends and patterns that would not be apparent to humans. For instance, for e-commerce websites like Amazon and Flipkart, it serves to understand the browsing behaviors and purchase histories of its users to help cater to the right products, deals, and reminders relevant to them. It uses the results to reveal relevant advertisements to them.

2. Continuous Improvement

We are continuously generating new data and when we provide this data to the Machine Learning model which helps it to upgrade with time and increase its performance and accuracy. We can say it is like gaining experience as they keep improving in accuracy and efficiency. This lets them make better decisions.

3. Handling multidimensional and multi-variety data

Machine Learning algorithms are good at handling data that are multidimensional and multi-variety, and they can do this in dynamic or uncertain environments.

4. Wide Applications

You could be an e-tailer or a healthcare provider and make Machine Learning work for you. Where it does apply, it holds the capability to help deliver a much more personal experience to customers while also targeting the right customers.

Disadvantages of Machine Learning

1. Data Acquisition

Machine Learning requires a massive amount of data sets to train on, and these should be inclusive/unbiased, and of good quality. There can also be times where we must wait for new data to be generated.

2. Time and Resources

Machine Learning needs enough time to let the algorithms learn and develop enough to fulfill their purpose with a considerable amount of accuracy and relevancy. It also needs massive resources to function. This can mean additional requirements of computer power for you.

3. Interpretation of Results

Another major challenge is the ability to accurately interpret results generated by the algorithms. You must also carefully choose the algorithms for your purpose. Sometimes, based on some analysis you might select an algorithm but it is not necessary that this model is best for the problem.

4. High error-susceptibility

Machine Learning is autonomous but highly susceptible to errors. Suppose you train an algorithm with data sets small enough to not be inclusive. You end up with biased predictions coming from a biased training set. This leads to irrelevant advertisements being displayed to customers. In the case of Machine Learning, such blunders can set off a chain of errors that can go undetected for long periods of time. And when they do get noticed, it takes quite some time to recognize the source of the issue, and even longer to correct it.

Future of Machine Learning

Machine Learning can be a competitive advantage to any company, be it a top MNC or a startup. As things that are currently being done manually will be done tomorrow by machines. With the introduction of projects such as self-driving cars, Sophia(a humanoid robot developed by Hong Kong-based company Hanson Robotics) we have already started a glimpse of what the future can be. The Machine Learning revolution will stay with us for long and so will be the future of Machine Learning.

Machine Learning Tutorial FAQs

How do I start learning Machine Learning?

You first need to start with the basics. You need to understand the prerequisites, which include learning Linear Algebra and Multivariate Calculus, Statistics, and Python. Then you need to learn several ML concepts, which include terminology of Machine Learning, types of Machine Learning, and Resources of Machine Learning. The third step is taking part in competitions. You can also take up a free online statistics for machine learning course and understand the foundational concepts.

Is Machine Learning easy for beginners? 

Machine Learning is not the easiest. The difficulty in learning Machine Learning is the debugging problem. However, if you study the right resources, you will be able to learn Machine Learning without any hassles.

What is a simple example of Machine Learning? 

Recommendation Engines (Netflix); Sorting, tagging and categorizing photos (Yelp); Customer Lifetime Value (Asos); Self-Driving Cars (Waymo); Education (Duolingo); Determining Credit Worthiness (Deserve); Patient Sickness Predictions (KenSci); and Targeted Emails (Optimail).

Can I learn Machine Learning in 3 months? 

Machine Learning is vast and consists of several things. Therefore, it will take you around six months to learn it, provided you spend at least 5-6 days every day. Also, the time taken to learn Machine Learning depends a lot on your mathematical and analytical skills.

Does Machine Learning require coding? 

If you are learning traditional Machine Learning, it would require you to know software programming as it will help you to write machine learning algorithms. However, through some online educational platforms, you do not need to know coding to learn Machine Learning.

Is Machine Learning a good career? 

Machine Learning is one of the best careers at present. Whether it is for the current demand, job, and salary growth, Machine Learning Engineer is one of the best profiles. You need to be very good at data, automation, and algorithms.

Can I learn Machine Learning without Python? 

To learn Machine Learning, you need to have some basic knowledge of Python. A version of Python that is supported by all Operating Systems such as Windows, Linux, etc., is Anaconda. It offers an overall package for machine learning, including matplotlib, scikit-learn, and NumPy.

Where can I practice Machine Learning? 

The online platforms where you can practice Machine Learning include CloudXLab, Google Colab, Kaggle, MachineHack, and OpenML.

Where can I learn Machine Learning for free?

You can learn the basics of Machine Learning from online platforms like Great Learning. You can enroll in the Beginners Machine Learning course and get the certificate for free. The course is easy and perfect for beginners to start with.

---

Original article source at: https://www.mygreatlearning.com

#machine-learning 

Before Predicting Gold: Exploring ETF’s Tracking Error

“ As humans, we are attracted to the colorful leaves and sometimes pay little attention to the roots, the stem, and the branches that are essential to the tree’s life. This is comparable to doing a financial model where a novice Quant rushes to the modeling process without spending sufficient time exploring the intricate details of the data”.

LBMA PM Fix & GLD ETF from Notebook

In this article, I will share my personal experience in doing a research project for Global Precious Metals. My firm is a niche player in the precious metals market, offering global end-to-end service from sales and trading to storing and delivering precious metals to prospective clients. Their goal was to ‘bring some quantitative practice to old school physical work’. I was to take part in this revolution and help the firm realize a systematic way to allocate money to gold. The firm might not be familiar with deriving the mathematical cost function of a regression model, but they know the mechanics of the gold industry. I was always astonished by the heavy flow of information and findings revolving this polarized asset, which kindled my exploration from the very first day.I told myself “forget about modeling, it’s time to learn about gold”.There is a plethora of benefits and caveats of using gold ETF as an alternative to gold spot prices for modeling. I will also allude to the notion of fund tracking errors and underlying factors that caused it. Tracking error exists in every fund that tracks a benchmark. Because it exists, one has to question whether or not the gold ETF would represent the genuine value of gold. It introduces caution to not only gold modelers but also investors. This article will not go through the deep mechanism of tracking errors, but it will guide you through my resolve to it.The article will address the following considerations before developing a model for gold:

• Common gold assets to based our model on and why ETFs are a feasible choice_Calculating/Visualizing tracking errors for different gold ETFs_Verifying factors that affect the tracking error of gold ETFsChoosing a gold ETF based on tracking error

A Little Detail on Gold Choices

So we want to create an investment model for gold. The first question is which gold asset do we model with. Some of the asset choices are:

1. Physical gold: The London Bullion OTC market (LBMA) offers investors gold as a tangible asset. Holding gold this way can sustain long term wealth without any counter party risks. Even though LBMA is based in London, the market can be traded from across the globe. The LBMA website holds some useful information on setting prices and trading terms.Exchange futures contracts: Futures contract traders are composed of mainly hedgers and speculators. Hedgers would buy/sell the present contract price of gold they wish to hold (to be delivered in future) and hedge the risk of the price rising/falling before the expiry date. Speculators conduct trades in order to profit from their speculation of price movement, as their goal is not to hold physical gold itself.Gold Tracking Exchange Traded Fund (ETF): These type of ETFs track the returns of spot gold prices by holding physical gold and futures contracts. Investors would invest in the shares of the ETF. Futures speculators and ETF investors are similar because they trade gold without holding gold. There are other gold ETFs that does not gain exposure to the commodity directly, instead investing in companies that specializes in gold.

So which Gold to model?

Assume your objective is to invest and actually store the gold. Allocation to gold will be re-balanced weekly. We decided to use an ML model to predict the direction of one week returns using daily observations.

Using futures prices are out of the question because we are not hedging nor speculating on the futures price. We can base the model on the LBMA spot price, but this method conveys some issues when constructing our feature set.

#linear-regression #gold #error-tracking #data analysis

Test_cov_console: Flutter Console Coverage Test

Flutter Console Coverage Test

This small dart tools is used to generate Flutter Coverage Test report to console

How to install

Add a line like this to your package's pubspec.yaml (and run an implicit flutter pub get):

dev_dependencies:
  test_cov_console: ^0.2.2

How to run

run the following command to make sure all flutter library is up-to-date

flutter pub get
Running "flutter pub get" in coverage...                            0.5s

run the following command to generate lcov.info on coverage directory

flutter test --coverage
00:02 +1: All tests passed!

run the tool to generate report from lcov.info

flutter pub run test_cov_console
---------------------------------------------|---------|---------|---------|-------------------|
File                                         |% Branch | % Funcs | % Lines | Uncovered Line #s |
---------------------------------------------|---------|---------|---------|-------------------|
lib/src/                                     |         |         |         |                   |
 print_cov.dart                              |  100.00 |  100.00 |   88.37 |...,149,205,206,207|
 print_cov_constants.dart                    |    0.00 |    0.00 |    0.00 |    no unit testing|
lib/                                         |         |         |         |                   |
 test_cov_console.dart                       |    0.00 |    0.00 |    0.00 |    no unit testing|
---------------------------------------------|---------|---------|---------|-------------------|
 All files with unit testing                 |  100.00 |  100.00 |   88.37 |                   |
---------------------------------------------|---------|---------|---------|-------------------|

Optional parameter

If not given a FILE, "coverage/lcov.info" will be used.
-f, --file=<FILE>                      The target lcov.info file to be reported
-e, --exclude=<STRING1,STRING2,...>    A list of contains string for files without unit testing
                                       to be excluded from report
-l, --line                             It will print Lines & Uncovered Lines only
                                       Branch & Functions coverage percentage will not be printed
-i, --ignore                           It will not print any file without unit testing
-m, --multi                            Report from multiple lcov.info files
-c, --csv                              Output to CSV file
-o, --output=<CSV-FILE>                Full path of output CSV file
                                       If not given, "coverage/test_cov_console.csv" will be used
-t, --total                            Print only the total coverage
                                       Note: it will ignore all other option (if any), except -m
-p, --pass=<MINIMUM>                   Print only the whether total coverage is passed MINIMUM value or not
                                       If the value >= MINIMUM, it will print PASSED, otherwise FAILED
                                       Note: it will ignore all other option (if any), except -m
-h, --help                             Show this help

example run the tool with parameters

flutter pub run test_cov_console --file=coverage/lcov.info --exclude=_constants,_mock
---------------------------------------------|---------|---------|---------|-------------------|
File                                         |% Branch | % Funcs | % Lines | Uncovered Line #s |
---------------------------------------------|---------|---------|---------|-------------------|
lib/src/                                     |         |         |         |                   |
 print_cov.dart                              |  100.00 |  100.00 |   88.37 |...,149,205,206,207|
lib/                                         |         |         |         |                   |
 test_cov_console.dart                       |    0.00 |    0.00 |    0.00 |    no unit testing|
---------------------------------------------|---------|---------|---------|-------------------|
 All files with unit testing                 |  100.00 |  100.00 |   88.37 |                   |
---------------------------------------------|---------|---------|---------|-------------------|

report for multiple lcov.info files (-m, --multi)

It support to run for multiple lcov.info files with the followings directory structures:
1. No root module
<root>/<module_a>
<root>/<module_a>/coverage/lcov.info
<root>/<module_a>/lib/src
<root>/<module_b>
<root>/<module_b>/coverage/lcov.info
<root>/<module_b>/lib/src
...
2. With root module
<root>/coverage/lcov.info
<root>/lib/src
<root>/<module_a>
<root>/<module_a>/coverage/lcov.info
<root>/<module_a>/lib/src
<root>/<module_b>
<root>/<module_b>/coverage/lcov.info
<root>/<module_b>/lib/src
...
You must run test_cov_console on <root> dir, and the report would be grouped by module, here is
the sample output for directory structure 'with root module':
flutter pub run test_cov_console --file=coverage/lcov.info --exclude=_constants,_mock --multi
---------------------------------------------|---------|---------|---------|-------------------|
File                                         |% Branch | % Funcs | % Lines | Uncovered Line #s |
---------------------------------------------|---------|---------|---------|-------------------|
lib/src/                                     |         |         |         |                   |
 print_cov.dart                              |  100.00 |  100.00 |   88.37 |...,149,205,206,207|
lib/                                         |         |         |         |                   |
 test_cov_console.dart                       |    0.00 |    0.00 |    0.00 |    no unit testing|
---------------------------------------------|---------|---------|---------|-------------------|
 All files with unit testing                 |  100.00 |  100.00 |   88.37 |                   |
---------------------------------------------|---------|---------|---------|-------------------|
---------------------------------------------|---------|---------|---------|-------------------|
File - module_a -                            |% Branch | % Funcs | % Lines | Uncovered Line #s |
---------------------------------------------|---------|---------|---------|-------------------|
lib/src/                                     |         |         |         |                   |
 print_cov.dart                              |  100.00 |  100.00 |   88.37 |...,149,205,206,207|
lib/                                         |         |         |         |                   |
 test_cov_console.dart                       |    0.00 |    0.00 |    0.00 |    no unit testing|
---------------------------------------------|---------|---------|---------|-------------------|
 All files with unit testing                 |  100.00 |  100.00 |   88.37 |                   |
---------------------------------------------|---------|---------|---------|-------------------|
---------------------------------------------|---------|---------|---------|-------------------|
File - module_b -                            |% Branch | % Funcs | % Lines | Uncovered Line #s |
---------------------------------------------|---------|---------|---------|-------------------|
lib/src/                                     |         |         |         |                   |
 print_cov.dart                              |  100.00 |  100.00 |   88.37 |...,149,205,206,207|
lib/                                         |         |         |         |                   |
 test_cov_console.dart                       |    0.00 |    0.00 |    0.00 |    no unit testing|
---------------------------------------------|---------|---------|---------|-------------------|
 All files with unit testing                 |  100.00 |  100.00 |   88.37 |                   |
---------------------------------------------|---------|---------|---------|-------------------|

Output to CSV file (-c, --csv, -o, --output)

flutter pub run test_cov_console -c --output=coverage/test_coverage.csv

#### sample CSV output file:
File,% Branch,% Funcs,% Lines,Uncovered Line #s
lib/,,,,
test_cov_console.dart,0.00,0.00,0.00,no unit testing
lib/src/,,,,
parser.dart,100.00,100.00,97.22,"97"
parser_constants.dart,100.00,100.00,100.00,""
print_cov.dart,100.00,100.00,82.91,"29,49,51,52,171,174,177,180,183,184,185,186,187,188,279,324,325,387,388,389,390,391,392,393,394,395,398"
print_cov_constants.dart,0.00,0.00,0.00,no unit testing
All files with unit testing,100.00,100.00,86.07,""

Installing

Use this package as an executable

Install it

You can install the package from the command line:

dart pub global activate test_cov_console

Use it

The package has the following executables:

$ test_cov_console

Use this package as a library

Depend on it

Run this command:

With Dart:

 $ dart pub add test_cov_console

With Flutter:

 $ flutter pub add test_cov_console

This will add a line like this to your package's pubspec.yaml (and run an implicit dart pub get):

dependencies:
  test_cov_console: ^0.2.2

Alternatively, your editor might support dart pub get or flutter pub get. Check the docs for your editor to learn more.

Import it

Now in your Dart code, you can use:

import 'package:test_cov_console/test_cov_console.dart';

example/lib/main.dart

import 'package:flutter/material.dart';

void main() {
  runApp(MyApp());
}

class MyApp extends StatelessWidget {
  // This widget is the root of your application.
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        // This is the theme of your application.
        //
        // Try running your application with "flutter run". You'll see the
        // application has a blue toolbar. Then, without quitting the app, try
        // changing the primarySwatch below to Colors.green and then invoke
        // "hot reload" (press "r" in the console where you ran "flutter run",
        // or simply save your changes to "hot reload" in a Flutter IDE).
        // Notice that the counter didn't reset back to zero; the application
        // is not restarted.
        primarySwatch: Colors.blue,
        // This makes the visual density adapt to the platform that you run
        // the app on. For desktop platforms, the controls will be smaller and
        // closer together (more dense) than on mobile platforms.
        visualDensity: VisualDensity.adaptivePlatformDensity,
      ),
      home: MyHomePage(title: 'Flutter Demo Home Page'),
    );
  }
}

class MyHomePage extends StatefulWidget {
  MyHomePage({Key? key, required this.title}) : super(key: key);

  // This widget is the home page of your application. It is stateful, meaning
  // that it has a State object (defined below) that contains fields that affect
  // how it looks.

  // This class is the configuration for the state. It holds the values (in this
  // case the title) provided by the parent (in this case the App widget) and
  // used by the build method of the State. Fields in a Widget subclass are
  // always marked "final".

  final String title;

  @override
  _MyHomePageState createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  int _counter = 0;

  void _incrementCounter() {
    setState(() {
      // This call to setState tells the Flutter framework that something has
      // changed in this State, which causes it to rerun the build method below
      // so that the display can reflect the updated values. If we changed
      // _counter without calling setState(), then the build method would not be
      // called again, and so nothing would appear to happen.
      _counter++;
    });
  }

  @override
  Widget build(BuildContext context) {
    // This method is rerun every time setState is called, for instance as done
    // by the _incrementCounter method above.
    //
    // The Flutter framework has been optimized to make rerunning build methods
    // fast, so that you can just rebuild anything that needs updating rather
    // than having to individually change instances of widgets.
    return Scaffold(
      appBar: AppBar(
        // Here we take the value from the MyHomePage object that was created by
        // the App.build method, and use it to set our appbar title.
        title: Text(widget.title),
      ),
      body: Center(
        // Center is a layout widget. It takes a single child and positions it
        // in the middle of the parent.
        child: Column(
          // Column is also a layout widget. It takes a list of children and
          // arranges them vertically. By default, it sizes itself to fit its
          // children horizontally, and tries to be as tall as its parent.
          //
          // Invoke "debug painting" (press "p" in the console, choose the
          // "Toggle Debug Paint" action from the Flutter Inspector in Android
          // Studio, or the "Toggle Debug Paint" command in Visual Studio Code)
          // to see the wireframe for each widget.
          //
          // Column has various properties to control how it sizes itself and
          // how it positions its children. Here we use mainAxisAlignment to
          // center the children vertically; the main axis here is the vertical
          // axis because Columns are vertical (the cross axis would be
          // horizontal).
          mainAxisAlignment: MainAxisAlignment.center,
          children: <Widget>[
            Text(
              'You have pushed the button this many times:',
            ),
            Text(
              '$_counter',
              style: Theme.of(context).textTheme.headline4,
            ),
          ],
        ),
      ),
      floatingActionButton: FloatingActionButton(
        onPressed: _incrementCounter,
        tooltip: 'Increment',
        child: Icon(Icons.add),
      ), // This trailing comma makes auto-formatting nicer for build methods.
    );
  }
}

Author: DigitalKatalis
Source Code: https://github.com/DigitalKatalis/test_cov_console 
License: BSD-3-Clause license

#flutter #dart #test 

Easy Activity Tracking for Models, Similar to Github's Public Activity

PublicActivity

public_activity provides easy activity tracking for your ActiveRecord, Mongoid 3 and MongoMapper models in Rails 3 and 4.

Simply put: it can record what happens in your application and gives you the ability to present those recorded activities to users - in a similar way to how GitHub does it.

!! WARNING: README for unreleased version below. !!

You probably don't want to read the docs for this unreleased version 2.0.

For the stable 1.5.X readme see: https://github.com/chaps-io/public_activity/blob/1-5-stable/README.md

About

Here is a simple example showing what this gem is about:

Tutorials

Screencast

Ryan Bates made a great screencast describing how to integrate Public Activity.

Tutorial

A great step-by-step guide on implementing activity feeds using public_activity by Ilya Bodrov.

Online demo

You can see an actual application using this gem here: http://public-activity-example.herokuapp.com/feed

The source code of the demo is hosted here: https://github.com/pokonski/activity_blog

Setup

Gem installation

You can install public_activity as you would any other gem:

gem install public_activity

or in your Gemfile:

gem 'public_activity'

Database setup

By default public_activity uses Active Record. If you want to use Mongoid or MongoMapper as your backend, create an initializer file in your Rails application with the corresponding code inside:

For Mongoid:

# config/initializers/public_activity.rb
PublicActivity.configure do |config|
  config.orm = :mongoid
end

For MongoMapper:

# config/initializers/public_activity.rb
PublicActivity.configure do |config|
  config.orm = :mongo_mapper
end

(ActiveRecord only) Create migration for activities and migrate the database (in your Rails project):

rails g public_activity:migration
rake db:migrate

Model configuration

Include PublicActivity::Model and add tracked to the model you want to keep track of:

For ActiveRecord:

class Article < ActiveRecord::Base
  include PublicActivity::Model
  tracked
end

For Mongoid:

class Article
  include Mongoid::Document
  include PublicActivity::Model
  tracked
end

For MongoMapper:

class Article
  include MongoMapper::Document
  include PublicActivity::Model
  tracked
end

And now, by default create/update/destroy activities are recorded in activities table. This is all you need to start recording activities for basic CRUD actions.

Optional: If you don't need #tracked but still want the comfort of #create_activity, you can include only the lightweight Common module instead of Model.

Custom activities

You can trigger custom activities by setting all your required parameters and triggering create_activity on the tracked model, like this:

@article.create_activity key: 'article.commented_on', owner: current_user

See this entry http://rubydoc.info/gems/public_activity/PublicActivity/Common:create_activity for more details.

Displaying activities

To display them you simply query the PublicActivity::Activity model:

# notifications_controller.rb
def index
  @activities = PublicActivity::Activity.all
end

And in your views:

<%= render_activities(@activities) %>

Note: render_activities is an alias for render_activity and does the same.

Layouts

You can also pass options to both activity#render and #render_activity methods, which are passed deeper to the internally used render_partial method. A useful example would be to render activities wrapped in layout, which shares common elements of an activity, like a timestamp, owner's avatar etc:

<%= render_activities(@activities, layout: :activity) %>

The activity will be wrapped with the app/views/layouts/_activity.html.erb layout, in the above example.

Important: please note that layouts for activities are also partials. Hence the _ prefix.

Locals

Sometimes, it's desirable to pass additional local variables to partials. It can be done this way:

<%= render_activity(@activity, locals: {friends: current_user.friends}) %>

Note: Before 1.4.0, one could pass variables directly to the options hash for #render_activity and access it from activity parameters. This functionality is retained in 1.4.0 and later, but the :locals method is preferred, since it prevents bugs from shadowing variables from activity parameters in the database.

Activity views

public_activity looks for views in app/views/public_activity.

For example, if you have an activity with :key set to "activity.user.changed_avatar", the gem will look for a partial in app/views/public_activity/user/_changed_avatar.html.(|erb|haml|slim|something_else).

Hint: the "activity." prefix in :key is completely optional and kept for backwards compatibility, you can skip it in new projects.

If you would like to fallback to a partial, you can utilize the fallback parameter to specify the path of a partial to use when one is missing:

<%= render_activity(@activity, fallback: 'default') %>

When used in this manner, if a partial with the specified :key cannot be located it will use the partial defined in the fallback instead. In the example above this would resolve to public_activity/_default.html.(|erb|haml|slim|something_else).

If a view file does not exist then ActionView::MisingTemplate will be raised. If you wish to fallback to the old behaviour and use an i18n based translation in this situation you can specify a :fallback parameter of text to fallback to this mechanism like such:

<%= render_activity(@activity, fallback: :text) %>

i18n

Translations are used by the #text method, to which you can pass additional options in form of a hash. #render method uses translations when view templates have not been provided. You can render pure i18n strings by passing {display: :i18n} to #render_activity or #render.

Translations should be put in your locale .yml files. To render pure strings from I18n Example structure:

activity:
  article:
    create: 'Article has been created'
    update: 'Someone has edited the article'
    destroy: 'Some user removed an article!'

This structure is valid for activities with keys "activity.article.create" or "article.create". As mentioned before, "activity." part of the key is optional.

Testing

For RSpec you can first disable public_activity and add require helper methods in the rails_helper.rb with:

#rails_helper.rb
require 'public_activity/testing'

PublicActivity.enabled = false

In your specs you can then blockwise decide whether to turn public_activity on or off.

# file_spec.rb
PublicActivity.with_tracking do
  # your test code goes here
end

PublicActivity.without_tracking do
  # your test code goes here
end

Documentation

For more documentation go here

Common examples

Set the Activity's owner to current_user by default

You can set up a default value for :owner by doing this:

1. Include PublicActivity::StoreController in your ApplicationController like this:

class ApplicationController < ActionController::Base
  include PublicActivity::StoreController
end

1. Use Proc in :owner attribute for tracked class method in your desired model. For example:

class Article < ActiveRecord::Base
  tracked owner: Proc.new{ |controller, model| controller.current_user }
end

Note: current_user applies to Devise, if you are using a different authentication gem or your own code, change the current_user to a method you use.

Disable tracking for a class or globally

If you need to disable tracking temporarily, for example in tests or db/seeds.rb then you can use PublicActivity.enabled= attribute like below:

# Disable p_a globally
PublicActivity.enabled = false

# Perform some operations that would normally be tracked by p_a:
Article.create(title: 'New article')

# Switch it back on
PublicActivity.enabled = true

You can also disable public_activity for a specific class:

# Disable p_a for Article class
Article.public_activity_off

# p_a will not do anything here:
@article = Article.create(title: 'New article')

# But will be enabled for other classes:
# (creation of the comment will be recorded if you are tracking the Comment class)
@article.comments.create(body: 'some comment!')

# Enable it again for Article:
Article.public_activity_on

Create custom activities

Besides standard, automatic activities created on CRUD actions on your model (deactivatable), you can post your own activities that can be triggered without modifying the tracked model. There are a few ways to do this, as PublicActivity gives three tiers of options to be set.

Instant options

Because every activity needs a key (otherwise: NoKeyProvided is raised), the shortest and minimal way to post an activity is:

@user.create_activity :mood_changed
# the key of the action will be user.mood_changed
@user.create_activity action: :mood_changed # this is exactly the same as above

Besides assigning your key (which is obvious from the code), it will take global options from User class (given in #tracked method during class definition) and overwrite them with instance options (set on @user by #activity method). You can read more about options and how PublicActivity inherits them for you here.

Note the action parameter builds the key like this: "#{model_name}.#{action}". You can read further on options for #create_activity here.

To provide more options, you can do:

@user.create_activity action: 'poke', parameters: {reason: 'bored'}, recipient: @friend, owner: current_user

In this example, we have provided all the things we could for a standard Activity.

Use custom fields on Activity

Besides the few fields that every Activity has (key, owner, recipient, trackable, parameters), you can also set custom fields. This could be very beneficial, as parameters are a serialized hash, which cannot be queried easily from the database. That being said, use custom fields when you know that you will set them very often and search by them (don't forget database indexes :) ).

Set owner and recipient based on associations

class Comment < ActiveRecord::Base
  include PublicActivity::Model
  tracked owner: :commenter, recipient: :commentee

  belongs_to :commenter, :class_name => "User"
  belongs_to :commentee, :class_name => "User"
end

Resolve parameters from a Symbol or Proc

class Post < ActiveRecord::Base
  include PublicActivity::Model
  tracked only: [:update], parameters: :tracked_values
  
  def tracked_values
   {}.tap do |hash|
     hash[:tags] = tags if tags_changed?
   end
  end
end

Setup

Skip this step if you are using ActiveRecord in Rails 4 or Mongoid

The first step is similar in every ORM available (except mongoid):

PublicActivity::Activity.class_eval do
  attr_accessible :custom_field
end

place this code under config/initializers/public_activity.rb, you have to create it first.

To be able to assign to that field, we need to move it to the mass assignment sanitizer's whitelist.

Migration

If you're using ActiveRecord, you will also need to provide a migration to add the actual field to the Activity. Taken from our tests:

class AddCustomFieldToActivities < ActiveRecord::Migration
  def change
    change_table :activities do |t|
      t.string :custom_field
    end
  end
end

Assigning custom fields

Assigning is done by the same methods that you use for normal parameters: #tracked, #create_activity. You can just pass the name of your custom variable and assign its value. Even better, you can pass it to #tracked to tell us how to harvest your data for custom fields so we can do that for you.

class Article < ActiveRecord::Base
  include PublicActivity::Model
  tracked custom_field: proc {|controller, model| controller.some_helper }
end

Help

If you need help with using public_activity please visit our discussion group and ask a question there:

https://groups.google.com/forum/?fromgroups#!forum/public-activity

Please do not ask general questions in the Github Issues.

---

Author: public-activity
Source code: https://github.com/public-activity/public_activity
License: MIT license

#ruby  #ruby-on-rails 

Immediate $olution to Fix AOL Blerk Error Code 5 with easy instructions

AOL Email is one of the leading web email services. It has a number of features who access easily at any place. Through this, you can easily share messages, documents or files, etc.AOL Blerk Error is not a big issue. It is a temporary error and it occurs when there is an issue in loading messages from the AOL server. If your mind is stuck, How to Resolve or Fix AOL Blerk Error Code 5? Here, In this article, we mentioned troubleshooting steps to fix AOL Blerk Error Code 5.

What are the causes of AOL Blerk Error Code 5?

AOL mail usually presents an AOL Blerk Error 5 after the AOL connection details have been entered. meaning. Your password and your username. This error is usually found in words! Or 'BLERK! Error 5 Authentication problem, 'Your sign-in has been received.

Some of the reasons for the error are as follows:
• Internet browser configuration problem

• Saved erroneous bookmark addresses

• browser cache or cookie

• An AOL Desktop Gold technical error.
How to Fix AOL Mail Blerk Error 5 in a Simple Way

This type of error is mostly due to your browser settings or the use of outdated, obsolete software. Users should remember that the steps to solve problems vary, depending on the browser you are using. Here are the steps to fix the mistake, check your browser and follow the steps.

Internet Explorer: Make sure you use the most recent web browser version. Open a new window and follow the “Tools> Web Options> Security> Internet Zone” thread. Activate ‘Safeguard Mode’ and follow the steps to include AOL Mail in the list of assured websites. Start the browser again to save changes and run Internet Explorer without additional information.
Firefox Mozilla: Open a new Firefox window and press Menu. To start the browser in safe mode, disable the add-on and choose the option to restart Firefox. You can see two options in the dialog box. Use the “Start in Safe Mode” option to disable all themes and extensions. The browser also turns off the hardware speed and resets the toolbar. You should be able to execute AOL mail when this happens.

Google Chrome: Update to the latest version of Chrome. Open the browser and go to the Advanced Options section. Go to ‘Security and Privacy’ and close the appropriate add-ons. Once the browsing history is deleted, the password, cookies saved and the cache will be cleared. Restart your system and try to log in to your AOL account with a new window.

Safari: Some pop-up windows block AOL mail when it comes to Safari and causes authentication issues. To fix the error, use Safari Security Preferences to enable the pop-up window and disable the security warning.

If you see, even when you change the required browser settings, the black error will not disappear, you can consult a skilled professional and see all the AOL email customer support numbers.

Get Connect to Fix Blerk Error Even After Clearing Cache & Cookies?
Somehow you can contact AOL technical support directly and get immediate help if you still get the error. Call +1(888)857-5157 to receive assistance from the AOL technical support team.

Source: https://email-expert247.blogspot.com/2021/04/immediate-olution-to-fix-aol-blerk.html “How to Resolve or Fix AOL Blerk Error Code 5”)**? Here, In this article, we mentioned troubleshooting steps to fix AOL Blerk Error Code 5.

What are the causes of AOL Blerk Error Code 5?

AOL mail usually presents an AOL Blerk Error 5 after the AOL connection details have been entered. meaning. Your password and your username. This error is usually found in words! Or 'BLERK! Error 5 Authentication problem, 'Your sign-in has been received.

Some of the reasons for the error are as follows:
• Internet browser configuration problem

• Saved erroneous bookmark addresses

• browser cache or cookie

• An AOL Desktop Gold technical error.
How to Fix AOL Mail Blerk Error 5 in a Simple Way

This type of error is mostly due to your browser settings or the use of outdated, obsolete software. Users should remember that the steps to solve problems vary, depending on the browser you are using. Here are the steps to fix the mistake, check your browser and follow the steps.

1. Internet Explorer: Make sure you use the most recent web browser version. Open a new window and follow the “Tools> Web Options> Security> Internet Zone” thread. Activate ‘Safeguard Mode’ and follow the steps to include AOL Mail in the list of assured websites. Start the browser again to save changes and run Internet Explorer without additional information.

2. Firefox Mozilla: Open a new Firefox window and press Menu. To start the browser in safe mode, disable the add-on and choose the option to restart Firefox. You can see two options in the dialog box. Use the “Start in Safe Mode” option to disable all themes and extensions. The browser also turns off the hardware speed and resets the toolbar. You should be able to execute AOL mail when this happens.

3. Google Chrome: Update to the latest version of Chrome. Open the browser and go to the Advanced Options section. Go to ‘Security and Privacy’ and close the appropriate add-ons. Once the browsing history is deleted, the password, cookies saved and the cache will be cleared. Restart your system and try to log in to your AOL account with a new window.

4. Safari: Some pop-up windows block AOL mail when it comes to Safari and causes authentication issues. To fix the error, use Safari Security Preferences to enable the pop-up window and disable the security warning.

If you see, even when you change the required browser settings, the black error will not disappear, you can consult a skilled professional and see all the AOL email customer support numbers.

Get Connect to Fix Blerk Error Even After Clearing Cache & Cookies?

Somehow you can contact AOL technical support directly and get immediate help if you still get the error. Call +1(888)857-5157 to receive assistance from the AOL technical support team.

Source: https://email-expert247.blogspot.com/2021/04/immediate-olution-to-fix-aol-blerk.html

#aol blerk error code 5 #aol blerk error 5 #aol mail blerk error code 5 #aol mail blerk error 5 #aol error code 5 #aol error 5