Grace  Lesch

Grace Lesch

1622729790

How We Learned to Stop Guessing and Love Low P-Values

How often should we measure performance? We could do so only when we make changes in the critical path, but [what if seemingly unrelated changes affect translation unit boundaries, and your compiler optimizations are affected?]\  What if you are consistently making changes to the critical path because you’re constantly trying to deliver better performance for your users? Getting into the business of predicting which changes will affect performance can be dangerous. In most cases, _you should be regularly testing for performance regressions. _ With regular commits from over 100 contributors, we’ve opted to measure this nightly, and for changes we expect to directly impact performance, we additionally measure and evaluate before committing.

#databases #p-values

What is GEEK

Buddha Community

How We Learned to Stop Guessing and Love Low P-Values
Grace  Lesch

Grace Lesch

1622729790

How We Learned to Stop Guessing and Love Low P-Values

How often should we measure performance? We could do so only when we make changes in the critical path, but [what if seemingly unrelated changes affect translation unit boundaries, and your compiler optimizations are affected?]\  What if you are consistently making changes to the critical path because you’re constantly trying to deliver better performance for your users? Getting into the business of predicting which changes will affect performance can be dangerous. In most cases, _you should be regularly testing for performance regressions. _ With regular commits from over 100 contributors, we’ve opted to measure this nightly, and for changes we expect to directly impact performance, we additionally measure and evaluate before committing.

#databases #p-values

Sofia  Maggio

Sofia Maggio

1649260800

Cl-online-learning: online Learning Algorithms

Cl-Online-Learning

A collection of machine learning algorithms for online linear classification written in Common Lisp.

Implemented algorithms

Binary classifier

  • Perceptron
  • AROW (Crammer, Koby, Alex Kulesza, and Mark Dredze. “Adaptive regularization of weight vectors.” Advances in neural information processing systems. 2009.)
  • SCW-I (Soft Confidence Weighted) (Wang, Jialei, Peilin Zhao, and Steven C. Hoi. “Exact Soft Confidence-Weighted Learning.” Proceedings of the 29th International Conference on Machine Learning (ICML-12). 2012.)
  • Logistic Regression with SGD or ADAM optimizer (Kingma, Diederik, and Jimmy Ba. “Adam: A method for stochastic optimization.” ICLR 2015)

Multiclass classifier

  • one-vs-rest ( K binary classifier required )
  • one-vs-one ( K*(K-1)/2 binary classifier required )

Command line tools

Installation

cl-online-learning is available from Quicklisp.

(ql:quickload :cl-online-learning)

When install from github repository,

cd ~/quicklisp/local-projects/
git clone https://github.com/masatoi/cl-online-learning.git

When using Roswell,

ros install masatoi/cl-online-learning

Usage

Prepare dataset

A data point is a pair of a class label (+1 or -1) and a input vector. Both of them have to be declared as single-float.

And dataset is represented as a sequence of data points. READ-DATA function is available to make a dataset from a sparse format used in LIBSVM (http://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/). This function requires the number of features of that dataset.

;; Number of features
(defparameter a1a-dim 123)

;; Read dataset from file
(defparameter a1a
  (clol.utils:read-data
   (merge-pathnames #P"t/dataset/a1a" (asdf:system-source-directory :cl-online-learning))
   a1a-dim))

;; A data point
(car a1a)

; (-1.0
;  . #(0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0
;     1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
;     0.0 0.0 1.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
;     1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0
;     1.0 0.0 1.0 1.0 0.0 0.0 0.0 1.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
;     0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
;     0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0))

Define learner

A learner object is just a struct, therefore their constructor is available to make it.

(defparameter arow-learner (clol:make-arow a1a-dim 10))

Update and Train

To update the model destructively with one data point, use an update function corresponding to the model type.

(clol:arow-update arow-learner
                  (cdar a1a)  ; input
                  (caar a1a)) ; label

TRAIN function can be used to learn the dataset collectively.

(clol:train arow-learner a1a)

It may be necessary to call this function several times until learning converges. For now, the convergence test has not been implemented yet.

Predict and Test

(clol:arow-predict arow-learner (cdar a1a))
; => -1.0

(clol:test arow-learner a1a)
; Accuracy: 84.85981%, Correct: 1362, Total: 1605

Multiclass classification

For multiclass data, the label of the data point is an integer representing the index of the class. READ-DATA function with MULTICLASS-P keyword option is available for make such a dataset.

(defparameter iris-dim 4)

; A dataset in which a same label appears consecutively need to be shuffled
(defparameter iris
  (clol.utils:shuffle-vector
   (coerce (clol.utils:read-data
            (merge-pathnames #P"t/dataset/iris.scale"
                             (asdf:system-source-directory :cl-online-learning))
            iris-dim :multiclass-p t)
	   'simple-vector)))

(defparameter iris-train (subseq iris 0 100))
(defparameter iris-test (subseq iris 100))

ONE-VS-REST and ONE-VS-ONE are available for multiclass classification by using multiple binary classifiers. In many cases, ONE-VS-ONE is more accurate, but it requires more computational resource as the number of classes increases.

;; Define model
(defparameter arow-1vs1
  (clol:make-one-vs-one iris-dim      ; Input data dimension
                        3             ; Number of class
                        'arow 0.1)) ; Binary classifier type and its parameters

;; Train and test model
(clol:train arow-1vs1 iris-train)
(clol:test  arow-1vs1 iris-test)
; Accuracy: 98.0%, Correct: 49, Total: 50

Sparse data

For sparse data (most elements are 0), the data point is a pair of a class label and a instance of SPARSE-VECTOR struct, and a learner with SPARSE- prefix is used. READ-DATA function with SPARSE-P keyword option is available for make such a dataset.

For example, news20.binary data has too high dimensional features to handle with normal learners. However, by using the sparse version, the learner can be trained with practical computational resources.

(defparameter news20.binary-dim 1355191)
(defparameter news20.binary (clol.utils:read-data "/path/to/news20.binary" news20.binary-dim :sparse-p t))

(defparameter news20.binary.arow (clol:make-sparse-arow news20.binary-dim 10))
(time (loop repeat 20 do (clol:train news20.binary.arow news20.binary)))
;; Evaluation took:
;;   1.527 seconds of real time
;;   1.526852 seconds of total run time (1.526852 user, 0.000000 system)
;;   100.00% CPU
;;   5,176,917,149 processor cycles
;;   11,436,032 bytes consed
(clol:test news20.binary.arow news20.binary)
; Accuracy: 99.74495%, Correct: 19945, Total: 19996

In a similar way, the sparse version learners are also available in multiclass classification.

(defparameter news20-dim 62060)
(defparameter news20-train (clol.utils:read-data "/path/to/news20.scale" news20-dim :sparse-p t :multiclass-p t))
(defparameter news20-test (clol.utils:read-data "/path/to/news20.t.scale" news20-dim :sparse-p t :multiclass-p t))
(defparameter news20-arow (clol:make-one-vs-rest news20-dim 20 'sparse-arow 10))
(loop repeat 12 do (clol:train news20-arow news20-train))
(clol:test news20-arow news20-test)
; Accuracy: 86.90208%, Correct: 3470, Total: 3993

Save/Restore model

For saving a learner model to a file or restoring from the model file, SAVE and RESTORE function are available respectively. For the above multiclass classification example, saving / restoring code would be:

;; Save
(clol:save arow-1vs1 #P"/tmp/iris.model")
;; Restore
(defparameter restored-learner (clol:restore #P"/tmp/iris.model"))

(clol:test restored-learner iris-test)
; Accuracy: 98.0%, Correct: 49, Total: 50

Author: masatoi
Source Code: https://github.com/masatoi/cl-online-learning
License: MIT License
#machine-learning #algorithm 

Rusty  Shanahan

Rusty Shanahan

1596247980

Learn to Read P-Value in English

When we start studying the concepts of probability and statistics, there are a few topics that require us to take a logical leap, often leaving us confused. In my earlier post, I have talked about one such topic, confidence interval. In this post, I will try to explain another such confusing topic, P-value. (Spoiler alert: No, it is not probability, but is related to probability)

After this post, you will be able to understand the correct interpretation of P-value and how the P-value leads to rejection/failure of rejection of hypothesis testing. Any mathematical equation or concept becomes beautiful if we learn how to read it in simple English and that is exactly what I am trying to do here.

I would assume that you are aware of the basics of hypothesis testing and conditional probability.

Conditional probability: P(it will rain today | sky is grey) read as probability that it will rain today given that sky is grey is 0.4 means that there is only a 4% chance that it will rain today if the sky has become grey.

Hypothesis testing: a brief overview

Real-world scenario

Imagine you are a data scientist at Expedia. You need to analyze what factors drive the new users to become loyal customers of Expedia. You do some initial exploratory analysis of the data. You find out that when a new user book through some promotional offer or deal, he/she tends to return to the website. So if a new user in his first visit uses some promotional offer to book a service in Expedia, they are more likely to come back than those first-time users who booked directly without using any deal.

Having studied a little about arts, you notice that these deals and offers are not eye-catching to the website visitors because those are displayed in blue color. You think that if the deals and offers are redesigned and highlighted in red color, this might increase the retention rate of first-time users. You recommend this to business. But rather than simple idea narration, to show how confident you are in your claim statistically. You perform the following steps:

  • Redesign the promotional content of the website in red color, and show this new version only to half of the new traffic to the website
  • The rest of the new users see the old version of the website, i.e., blue color.

If in a day 100,000 new users visit the website, imagine 50,000 sees the old version and rest of the 50,000 sees the new version

In the new version, Number of new users who used the promo and returned after their first use is 7700

In the old version, Number of new users who used the promo and returned after their first use is 7000

Here, you observe a 10% increase in the number of retained users when they used the new version of the website. But before you conclude that changing the website promo colors to red can increase the retention rate to 10%, you must be sure that this increase is not by any random chance. This increase might be due to the way you sampled or because in general users who used a new version that day do not know any other travel booking website or because of any other random reason. For this clarity, you run a hypothesis test.

Hypothesis testing methodology

  1. Choose one test statistic (retention percentage here)
  2. Formulate hypothesis

**_Null Hypothesis (Always status quo): _**There is no difference in the retention percentage of new users between the two versions. Retention percentage of new version = retention percentage of the old version. The difference observed is just random.

Alternative hypothesis: There is a difference between the retention percentage of the new version and the retention percentage of the old version.

#hypothesis-testing #data-science #statistics #p-value #machine-learning #deep learning

Jerad  Bailey

Jerad Bailey

1598891580

Google Reveals "What is being Transferred” in Transfer Learning

Recently, researchers from Google proposed the solution of a very fundamental question in the machine learning community — What is being transferred in Transfer Learning? They explained various tools and analyses to address the fundamental question.

The ability to transfer the domain knowledge of one machine in which it is trained on to another where the data is usually scarce is one of the desired capabilities for machines. Researchers around the globe have been using transfer learning in various deep learning applications, including object detection, image classification, medical imaging tasks, among others.

#developers corner #learn transfer learning #machine learning #transfer learning #transfer learning methods #transfer learning resources

sophia tondon

sophia tondon

1620898103

5 Latest Technology Trends of Machine Learning for 2021

Check out the 5 latest technologies of machine learning trends to boost business growth in 2021 by considering the best version of digital development tools. It is the right time to accelerate user experience by bringing advancement in their lifestyle.

#machinelearningapps #machinelearningdevelopers #machinelearningexpert #machinelearningexperts #expertmachinelearningservices #topmachinelearningcompanies #machinelearningdevelopmentcompany

Visit Blog- https://www.xplace.com/article/8743

#machine learning companies #top machine learning companies #machine learning development company #expert machine learning services #machine learning experts #machine learning expert