I Taught My Computer to Classify Chinese Calligraphy Styles

Motivation

As an international student studying in China, I’ve always been fascinated by the diversity of Chinese culture and history. Calligraphy (書) is considered as one of the four ancient Chinese arts, along with stringed musical instruments (琴), the board game “Go” (棋), and painting (畫). Calligraphy has also contributed to the development of many forms of art in China, such as ornate paperweights, ink stones, and seal carving.

There are multiple styles of calligraphy, which mainly belong to different dynasties. Each of them has its way of shaping and arranging the character. For this project, I picked four styles:

• Seal Script (篆書 zhuanshu)
• Cursive Script (草書 caoshu)
• Clerical Script (隸書 lishu)
• Standard Script (楷書 kaishu)

If you are interested, you can read more about these different styles here.

The Chinese character for dragon (龍) written in 4 different calligraphy styles

Collecting the Data

To build a calligraphy classifier, we’re going to need some examples of each style. However, I did some online search and could not find a decently made dataset for various calligraphy styles. So, I decided to create the dataset myself. Fortunately, creating my own dataset isn’t that hard, thanks to Google Images’ search functionality and some JavaScript snippets. Here’s how:

Search results for ‘zhuanshu’

1. Suppose you want to search for the ‘zhuanshu’ style. Go to Google Images and search for “篆書字帖網格”, this will give you the most relevant results.
2. Scroll down and click the ‘Show more results’ button to view more images.
3. Here’s where the magic happens. Press Ctrl+Shift+J in Windows/Linux and Cmd+Opt+J in Mac to bring up the JavaScript console window. The following code snippet will retrieve the URLs of each of the images:

let urls = Array.from(document.querySelectorAll('.rg_i')).map(el=> el.hasAttribute('data-src')?el.getAttribute('data-src'):el.getAttribute('data-iurl'));
	let hiddenElement = document.createElement('a');
	hiddenElement.href = 'data:text/csv;charset=utf-8,' + encodeURI(urls.filter((value, index, arr) => { return value != null}).join('\n'));
	hiddenElement.target = '_blank';
	hiddenElement.download = 'download.csv';
	hiddenElement.click();
view raw
google_images_scraper.js hosted with ❤ by GitHub

4. If successfully run, a CSV file containing the URLs for the images in your search results will be downloaded. By default, the filename is download.csv but you might want to rename the file from to zhuanshu.csv

5. Repeat the above steps for other styles. You might also want to put them into different folders and put them all inside a folder called train

Folder tree structure

6. Finally, you can use fast.ai’s download_images function to download the images:

path = Path('<path>')

	for file, folder in [('caoshu.csv', 'caoshu'), ('kaishu.csv', 'kaishu'), 
	                     ('lishu.csv', 'lishu'), ('zhuanshu.csv', 'zhuanshu')]:
	    dest = path/folder  # path + '/' + folder
	    dest.mkdir(parents=True, exist_ok=True)
	    download_images(path/file, dest)
view raw
download_images.py hosted with ❤ by GitHub

Alternatively, you can go to Baidu Images and use this snippet to automatically download the images you searched for.

Preparing the Data

We can import, split, and transform the data using fast.ai’s powerful ImageDataBunch object. After running the following code, our data is ready to be fitted into a model:

np.random.seed(42)

	data = ImageDataBunch.from_folder(
	    path,
	    valid_pct=0.2,
	    ds_tfms=get_transforms(do_flip=False),
	    size=224,
	    num_workers=4    
	)

	data.normalize(imagenet_stats)
view raw
import_split_transform.py hosted with ❤ by GitHub

Note that we split the data into training and validation set with an 80:20 ratio. The images are also resized to 224 pixels, which is usually a good value for image recognition tasks. Now that we have our data is imported, let’s look at some of the images:

data.show_batch(rows=3, figsize=(9,10))

A batch of images in the dataset

As we can see above, our dataset is rather ‘dirty’. Some of the images are not well-aligned and not properly cropped. But let’s quickly build the model and see how it performs.

Building the Model

We will be applying transfer learning and use ResNet-50 as our model. Run the following code to download the model along with the pre-trained weights:

learn = cnn_learner(data, models.resnet50, metrics=accuracy)

To train the layers, we can either use the fit or fit_one_cycle method. The fit method is the “normal” way of training a neural net with a constant learning rate. On the other hand, the fit_one_cycle method uses the ‘1 cycle policy’, which basically changes the learning rate over time to achieve better results.

#deep-learning #transfer-learning #machine-learning #art #data-science #deep learning

What is GEEK

Buddha Community

How to Predict Housing Prices with Linear Regression?

The final objective is to estimate the cost of a certain house in a Boston suburb. In 1970, the Boston Standard Metropolitan Statistical Area provided the information. To examine and modify the data, we will use several techniques such as data pre-processing and feature engineering. After that, we'll apply a statistical model like regression model to anticipate and monitor the real estate market.

Project Outline:

• EDA
• Feature Engineering
• Pick and Train a Model
• Interpret
• Conclusion

EDA

Before using a statistical model, the EDA is a good step to go through in order to:

• Recognize the data set
• Check to see if any information is missing.
• Find some outliers.
• To get more out of the data, add, alter, or eliminate some features.

Importing the Libraries

• Recognize the data set
• Check to see if any information is missing.
• Find some outliers.
• To get more out of the data, add, alter, or eliminate some features.

# Import the libraries #Dataframe/Numerical libraries import pandas as pd import numpy as np #Data visualization import plotly.express as px import matplotlib import matplotlib.pyplot as plt import seaborn as sns #Machine learning model from sklearn.linear_model import LinearRegression

Reading the Dataset with Pandas

#Reading the data path='./housing.csv' housing_df=pd.read_csv(path,header=None,delim_whitespace=True)

	CRIM	ZN	INDUS	CHAS	NOX	RM	AGE	DIS	RAD	TAX	PTRATIO	B	LSTAT	MEDV
0	0.00632	18.0	2.31	0	0.538	6.575	65.2	4.0900	1	296.0	15.3	396.90	4.98	24.0
1	0.02731	0.0	7.07	0	0.469	6.421	78.9	4.9671	2	242.0	17.8	396.90	9.14	21.6
2	0.02729	0.0	7.07	0	0.469	7.185	61.1	4.9671	2	242.0	17.8	392.83	4.03	34.7
3	0.03237	0.0	2.18	0	0.458	6.998	45.8	6.0622	3	222.0	18.7	394.63	2.94	33.4
4	0.06905	0.0	2.18	0	0.458	7.147	54.2	6.0622	3	222.0	18.7	396.90	5.33	36.2
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
501	0.06263	0.0	11.93	0	0.573	6.593	69.1	2.4786	1	273.0	21.0	391.99	9.67	22.4
502	0.04527	0.0	11.93	0	0.573	6.120	76.7	2.2875	1	273.0	21.0	396.90	9.08	20.6
503	0.06076	0.0	11.93	0	0.573	6.976	91.0	2.1675	1	273.0	21.0	396.90	5.64	23.9
504	0.10959	0.0	11.93	0	0.573	6.794	89.3	2.3889	1	273.0	21.0	393.45	6.48	22.0
505	0.04741	0.0	11.93	0	0.573	6.030	80.8	2.5050	1	273.0	21.0	396.90	7.88	11.9

Have a Look at the Columns

Crime: It refers to a town's per capita crime rate.

ZN: It is the percentage of residential land allocated for 25,000 square feet.

Indus: The amount of non-retail business lands per town is referred to as the indus.

CHAS: CHAS denotes whether or not the land is surrounded by a river.

NOX: The NOX stands for nitric oxide content (part per 10m)

RM: The average number of rooms per home is referred to as RM.

AGE: The percentage of owner-occupied housing built before 1940 is referred to as AGE.

DIS: Weighted distance to five Boston employment centers are referred to as dis.

RAD: Accessibility to radial highways index

TAX: The TAX columns denote the rate of full-value property taxes per $10,000 dollars.

B: B=1000(Bk — 0.63)2 is the outcome of the equation, where Bk is the proportion of blacks in each town.

PTRATIO: It refers to the student-to-teacher ratio in each community.

LSTAT: It refers to the population's lower socioeconomic status.

MEDV: It refers to the 1000-dollar median value of owner-occupied residences.

Data Preprocessing

# Check if there is any missing values. housing_df.isna().sum() CRIM       0 ZN         0 INDUS      0 CHAS       0 NOX        0 RM         0 AGE        0 DIS        0 RAD        0 TAX        0 PTRATIO    0 B          0 LSTAT      0 MEDV       0 dtype: int64

No missing values are found

We examine our data's mean, standard deviation, and percentiles.

housing_df.describe()

	CRIM	ZN	INDUS	CHAS	NOX	RM	AGE	DIS	RAD	TAX	PTRATIO	B	LSTAT	MEDV
count	506.000000	506.000000	506.000000	506.000000	506.000000	506.000000	506.000000	506.000000	506.000000	506.000000	506.000000	506.000000	506.000000	506.000000
mean	3.613524	11.363636	11.136779	0.069170	0.554695	6.284634	68.574901	3.795043	9.549407	408.237154	18.455534	356.674032	12.653063	22.532806
std	8.601545	23.322453	6.860353	0.253994	0.115878	0.702617	28.148861	2.105710	8.707259	168.537116	2.164946	91.294864	7.141062	9.197104
min	0.006320	0.000000	0.460000	0.000000	0.385000	3.561000	2.900000	1.129600	1.000000	187.000000	12.600000	0.320000	1.730000	5.000000
25%	0.082045	0.000000	5.190000	0.000000	0.449000	5.885500	45.025000	2.100175	4.000000	279.000000	17.400000	375.377500	6.950000	17.025000
50%	0.256510	0.000000	9.690000	0.000000	0.538000	6.208500	77.500000	3.207450	5.000000	330.000000	19.050000	391.440000	11.360000	21.200000
75%	3.677083	12.500000	18.100000	0.000000	0.624000	6.623500	94.075000	5.188425	24.000000	666.000000	20.200000	396.225000	16.955000	25.000000
max	88.976200	100.000000	27.740000	1.000000	0.871000	8.780000	100.000000	12.126500	24.000000	711.000000	22.000000	396.900000	37.970000	50.000000

The crime, area, sector, nitric oxides, 'B' appear to have multiple outliers at first look because the minimum and maximum values are so far apart. In the Age columns, the mean and the Q2(50 percentile) do not match.

We might double-check it by examining the distribution of each column.

Inferences

1. The rate of crime is rather low. The majority of values are in the range of 0 to 25. With a huge value and a value of zero.
2. The majority of residential land is zoned for less than 25,000 square feet. Land zones larger than 25,000 square feet represent a small portion of the dataset.
3. The percentage of non-retial commercial acres is mostly split between two ranges: 0-13 and 13-23.
4. The majority of the properties are bordered by the river, although a tiny portion of the data is not.
5. The content of nitrite dioxide has been trending lower from.3 to.7, with a little bump towards.8. It is permissible to leave a value in the range of 0.1–1.
6. The number of rooms tends to cluster around the average.
7. With time, the proportion of owner-occupied units rises.
8. As the number of weights grows, the weight distance between 5 employment centers reduces. It could indicate that individuals choose to live in new high-employment areas.
9. People choose to live in places with limited access to roadways (0-10). We have a 30th percentile outlier.
10. The majority of dwelling taxes are in the range of $200-450, with large outliers around $700,000.
11. The percentage of people with lower status tends to cluster around the median. The majority of persons are of lower social standing.

Because the model is overly generic, removing all outliers will underfit it. Keeping all outliers causes the model to overfit and become excessively accurate. The data's noise will be learned.

The approach is to establish a happy medium that prevents the model from becoming overly precise. When faced with a new set of data, however, they generalise well.

We'll keep numbers below 600 because there's a huge anomaly in the TAX column around 600.

new_df=housing_df[housing_df['TAX']<600]

Looking at the Distribution

The overall distribution, particularly the TAX, PTRATIO, and RAD, has improved slightly.

Correlation

Perfect correlation is denoted by the clear values. The medium correlation between the columns is represented by the reds, while the negative correlation is represented by the black.

With a value of 0.89, we can see that 'MEDV', which is the medium price we wish to anticipate, is substantially connected with the number of rooms 'RM'. The proportion of black people in area 'B' with a value of 0.19 is followed by the residential land 'ZN' with a value of 0.32 and the percentage of black people in area 'ZN' with a value of 0.32.

The metrics that are most connected with price will be plotted.

Feature Engineering

Feature Scaling

Gradient descent is aided by feature scaling, which ensures that all features are on the same scale. It makes locating the local optimum much easier.

Mean standardization is one strategy to employ. It substitutes (target-mean) for the target to ensure that the feature has a mean of nearly zero.

def standard(X):    '''Standard makes the feature 'X' have a zero mean'''    mu=np.mean(X) #mean    std=np.std(X) #standard deviation    sta=(X-mu)/std # mean normalization    return mu,std,sta     mu,std,sta=standard(X) X=sta X

	CRIM	ZN	INDUS	CHAS	NOX	RM	AGE	DIS	RAD	TAX	PTRATIO	B	LSTAT
0	-0.609129	0.092792	-1.019125	-0.280976	0.258670	0.279135	0.162095	-0.167660	-2.105767	-0.235130	-1.136863	0.401318	-0.933659
1	-0.575698	-0.598153	-0.225291	-0.280976	-0.423795	0.049252	0.648266	0.250975	-1.496334	-1.032339	-0.004175	0.401318	-0.219350
2	-0.575730	-0.598153	-0.225291	-0.280976	-0.423795	1.189708	0.016599	0.250975	-1.496334	-1.032339	-0.004175	0.298315	-1.096782
3	-0.567639	-0.598153	-1.040806	-0.280976	-0.532594	0.910565	-0.526350	0.773661	-0.886900	-1.327601	0.403593	0.343869	-1.283945
4	-0.509220	-0.598153	-1.040806	-0.280976	-0.532594	1.132984	-0.228261	0.773661	-0.886900	-1.327601	0.403593	0.401318	-0.873561
...	...	...	...	...	...	...	...	...	...	...	...	...	...
501	-0.519445	-0.598153	0.585220	-0.280976	0.604848	0.306004	0.300494	-0.936773	-2.105767	-0.574682	1.445666	0.277056	-0.128344
502	-0.547094	-0.598153	0.585220	-0.280976	0.604848	-0.400063	0.570195	-1.027984	-2.105767	-0.574682	1.445666	0.401318	-0.229652
503	-0.522423	-0.598153	0.585220	-0.280976	0.604848	0.877725	1.077657	-1.085260	-2.105767	-0.574682	1.445666	0.401318	-0.820331
504	-0.444652	-0.598153	0.585220	-0.280976	0.604848	0.606046	1.017329	-0.979587	-2.105767	-0.574682	1.445666	0.314006	-0.676095
505	-0.543685	-0.598153	0.585220	-0.280976	0.604848	-0.534410	0.715691	-0.924173	-2.105767	-0.574682	1.445666	0.401318	-0.435703

Choose and Train the Model

For the sake of the project, we'll apply linear regression.

Typically, we run numerous models and select the best one based on a particular criterion.

Linear regression is a sort of supervised learning model in which the response is continuous, as it relates to machine learning.

Form of Linear Regression

y= θX+θ1 or y= θ1+X1θ2 +X2θ3 + X3θ4

y is the target you will be predicting

0 is the coefficient

x is the input

We will Sklearn to develop and train the model

#Import the libraries to train the model from sklearn.model_selection import train_test_split from sklearn.linear_model import LinearRegression

Allow us to utilise the train/test method to learn a part of the data on one set and predict using another set using the train/test approach.

X_train,X_test,y_train,y_test=train_test_split(X,y,test_size=0.4) #Create and Train the model model=LinearRegression().fit(X_train,y_train) #Generate prediction predictions_test=model.predict(X_test) #Compute loss to evaluate the model coefficient= model.coef_ intercept=model.intercept_ print(coefficient,intercept) [7.22218258] 24.66379606613584

In this example, you will learn the model using below hypothesis:

Price= 24.85 + 7.18* Room

It is interpreted as:

For a decided price of a house:

A 7.18-unit increase in the price is connected with a growth in the number of rooms.

As a side note, this is an association, not a cause!

Interpretation

You will need a metric to determine whether our hypothesis was right. The RMSE approach will be used.

Root Means Square Error (RMSE) is defined as the square root of the mean of square error. The difference between the true and anticipated numbers called the error. It's popular because it can be expressed in y-units, which is the median price of a home in our scenario.

def rmse(predict,actual):    return np.sqrt(np.mean(np.square(predict - actual))) # Split the Data into train and test set X_train,X_test,y_train,y_test=train_test_split(X,y,test_size=0.4) #Create and Train the model model=LinearRegression().fit(X_train,y_train) #Generate prediction predictions_test=model.predict(X_test) #Compute loss to evaluate the model coefficient= model.coef_ intercept=model.intercept_ print(coefficient,intercept) loss=rmse(predictions_test,y_test) print('loss: ',loss) print(model.score(X_test,y_test)) #accuracy [7.43327725] 24.912055881970886 loss: 3.9673165450580714 0.7552661033654667 Loss will be 3.96

This means that y-units refer to the median value of occupied homes with 1000 dollars.

This will be less by 3960 dollars.

While learning the model you will have a high variance when you divide the data. Coefficient and intercept will vary. It's because when we utilized the train/test approach, we choose a set of data at random to place in either the train or test set. As a result, our theory will change each time the dataset is divided.

This problem can be solved using a technique called cross-validation.

Improvisation in the Model

With 'Forward Selection,' we'll iterate through each parameter to assist us choose the numbers characteristics to include in our model.

Forward Selection

1. Choose the most appropriate variable (in our case based on high correlation)
2. Add the next best variable to the model
3. Some predetermined conditions must meet.

We'll use a random state of 1 so that each iteration yields the same outcome.

cols=[] los=[] los_train=[] scor=[] i=0 while i < len(high_corr_var):    cols.append(high_corr_var[i])        # Select inputs variables    X=new_df[cols]        #mean normalization    mu,std,sta=standard(X)    X=sta        # Split the data into training and testing    X_train,X_test,y_train,y_test= train_test_split(X,y,random_state=1)        #fit the model to the training    lnreg=LinearRegression().fit(X_train,y_train)        #make prediction on the training test    prediction_train=lnreg.predict(X_train)        #make prediction on the testing test    prediction=lnreg.predict(X_test)        #compute the loss on train test    loss=rmse(prediction,y_test)    loss_train=rmse(prediction_train,y_train)    los_train.append(loss_train)    los.append(loss)        #compute the score    score=lnreg.score(X_test,y_test)    scor.append(score)        i+=1

We have a big 'loss' with a smaller collection of variables, yet our system will overgeneralize in this scenario. Although we have a reduced 'loss,' we have a large number of variables. However, if the model grows too precise, it may not generalize well to new data.

In order for our model to generalize well with another set of data, we might use 6 or 7 features. The characteristic chosen is descending based on how strong the price correlation is.

high_corr_var ['RM', 'ZN', 'B', 'CHAS', 'RAD', 'DIS', 'CRIM', 'NOX', 'AGE', 'TAX', 'INDUS', 'PTRATIO', 'LSTAT']

With 'RM' having a high price correlation and LSTAT having a negative price correlation.

# Create a list of features names feature_cols=['RM','ZN','B','CHAS','RAD','CRIM','DIS','NOX'] #Select inputs variables X=new_df[feature_cols] # Split the data into training and testing sets X_train,X_test,y_train,y_test= train_test_split(X,y, random_state=1) # feature engineering mu,std,sta=standard(X) X=sta # fit the model to the trainning data lnreg=LinearRegression().fit(X_train,y_train) # make prediction on the testing test prediction=lnreg.predict(X_test) # compute the loss loss=rmse(prediction,y_test) print('loss: ',loss) lnreg.score(X_test,y_test) loss: 3.212659865936143 0.8582338376696363

The test set yielded a loss of 3.21 and an accuracy of 85%.

Other factors, such as alpha, the learning rate at which our model learns, could still be tweaked to improve our model. Alternatively, return to the preprocessing section and working to increase the parameter distribution.

For more details regarding scraping real estate data you can contact Scraping Intelligence today

https://www.websitescraper.com/how-to-predict-housing-prices-with-linear-regression.php

Cloud Computing Vs Grid Computing

The similarity between cloud computing and grid computing is uncanny. The underlying concepts that make these two inherently different are actually so similar to one and another, which is responsible for creating a lot of confusion. Both cloud and grid computing aims to provide a similar kind of services to a large user base by sharing assets among an enormous pool of clients.

Both of these technologies are obviously network-based and are capable enough to sport multitasking. The availability of multitasking allows the users of either of the two services to use multiple applications at the same time. You are also not limited to the kind of applications that you can use. You are free to choose any number of applications that can accomplish any tasks that you want. Learn more about cloud computing applications.

#cloud computing #cloud computing vs grid computing #grid computing #cloud

Tsf GPU: Basic Multi-GPU Computation in TensorFlow

Setup

Refer to the setup instructions

Noted: Updated for multi-GPU support. Soon a 8-way GPU cluster to be tested on my own DeepLearning box.

This tutorial requires your machine to have 2 GPUs

• "/cpu:0": The CPU of your machine.
• "/gpu:0": The first GPU of your machine
• "/gpu:1": The second GPU of your machine
• For this example, we are using 2 GTX-980

import numpy as np
import tensorflow as tf
import datetime

#Processing Units logs
log_device_placement = True

#num of multiplications to perform
n = 10

# Example: compute A^n + B^n on 2 GPUs

# Create random large matrix
A = np.random.rand(1e4, 1e4).astype('float32')
B = np.random.rand(1e4, 1e4).astype('float32')

# Creates a graph to store results
c1 = []
c2 = []

# Define matrix power
def matpow(M, n):
    if n < 1: #Abstract cases where n < 1
        return M
    else:
        return tf.matmul(M, matpow(M, n-1))

# Single GPU computing

with tf.device('/gpu:0'):
    a = tf.constant(A)
    b = tf.constant(B)
    #compute A^n and B^n and store results in c1
    c1.append(matpow(a, n))
    c1.append(matpow(b, n))

with tf.device('/cpu:0'):
  sum = tf.add_n(c1) #Addition of all elements in c1, i.e. A^n + B^n

t1_1 = datetime.datetime.now()
with tf.Session(config=tf.ConfigProto(log_device_placement=log_device_placement)) as sess:
    # Runs the op.
    sess.run(sum)
t2_1 = datetime.datetime.now()

# Multi GPU computing
# GPU:0 computes A^n
with tf.device('/gpu:0'):
    #compute A^n and store result in c2
    a = tf.constant(A)
    c2.append(matpow(a, n))

#GPU:1 computes B^n
with tf.device('/gpu:1'):
    #compute B^n and store result in c2
    b = tf.constant(B)
    c2.append(matpow(b, n))

with tf.device('/cpu:0'):
  sum = tf.add_n(c2) #Addition of all elements in c2, i.e. A^n + B^n

t1_2 = datetime.datetime.now()
with tf.Session(config=tf.ConfigProto(log_device_placement=log_device_placement)) as sess:
    # Runs the op.
    sess.run(sum)
t2_2 = datetime.datetime.now()

print "Single GPU computation time: " + str(t2_1-t1_1)
print "Multi GPU computation time: " + str(t2_2-t1_2)

Single GPU computation time: 0:00:11.833497
Multi GPU computation time: 0:00:07.085913

Link; https://nbviewer.org/github/TarrySingh/Machine-Learning-Tutorials/blob/master/deep-learning/tensor-flow-examples/notebooks/4_multi_gpu/multigpu_basics.ipynb

#tensorflow  #python 

I Taught My Computer to Classify Chinese Calligraphy Styles

Motivation

As an international student studying in China, I’ve always been fascinated by the diversity of Chinese culture and history. Calligraphy (書) is considered as one of the four ancient Chinese arts, along with stringed musical instruments (琴), the board game “Go” (棋), and painting (畫). Calligraphy has also contributed to the development of many forms of art in China, such as ornate paperweights, ink stones, and seal carving.

There are multiple styles of calligraphy, which mainly belong to different dynasties. Each of them has its way of shaping and arranging the character. For this project, I picked four styles:

• Seal Script (篆書 zhuanshu)
• Cursive Script (草書 caoshu)
• Clerical Script (隸書 lishu)
• Standard Script (楷書 kaishu)

If you are interested, you can read more about these different styles here.

The Chinese character for dragon (龍) written in 4 different calligraphy styles

Collecting the Data

To build a calligraphy classifier, we’re going to need some examples of each style. However, I did some online search and could not find a decently made dataset for various calligraphy styles. So, I decided to create the dataset myself. Fortunately, creating my own dataset isn’t that hard, thanks to Google Images’ search functionality and some JavaScript snippets. Here’s how:

Search results for ‘zhuanshu’

1. Suppose you want to search for the ‘zhuanshu’ style. Go to Google Images and search for “篆書字帖網格”, this will give you the most relevant results.
2. Scroll down and click the ‘Show more results’ button to view more images.
3. Here’s where the magic happens. Press Ctrl+Shift+J in Windows/Linux and Cmd+Opt+J in Mac to bring up the JavaScript console window. The following code snippet will retrieve the URLs of each of the images:

let urls = Array.from(document.querySelectorAll('.rg_i')).map(el=> el.hasAttribute('data-src')?el.getAttribute('data-src'):el.getAttribute('data-iurl'));
	let hiddenElement = document.createElement('a');
	hiddenElement.href = 'data:text/csv;charset=utf-8,' + encodeURI(urls.filter((value, index, arr) => { return value != null}).join('\n'));
	hiddenElement.target = '_blank';
	hiddenElement.download = 'download.csv';
	hiddenElement.click();
view raw
google_images_scraper.js hosted with ❤ by GitHub

4. If successfully run, a CSV file containing the URLs for the images in your search results will be downloaded. By default, the filename is download.csv but you might want to rename the file from to zhuanshu.csv

5. Repeat the above steps for other styles. You might also want to put them into different folders and put them all inside a folder called train

Folder tree structure

6. Finally, you can use fast.ai’s download_images function to download the images:

path = Path('<path>')

	for file, folder in [('caoshu.csv', 'caoshu'), ('kaishu.csv', 'kaishu'), 
	                     ('lishu.csv', 'lishu'), ('zhuanshu.csv', 'zhuanshu')]:
	    dest = path/folder  # path + '/' + folder
	    dest.mkdir(parents=True, exist_ok=True)
	    download_images(path/file, dest)
view raw
download_images.py hosted with ❤ by GitHub

Alternatively, you can go to Baidu Images and use this snippet to automatically download the images you searched for.

Preparing the Data

We can import, split, and transform the data using fast.ai’s powerful ImageDataBunch object. After running the following code, our data is ready to be fitted into a model:

np.random.seed(42)

	data = ImageDataBunch.from_folder(
	    path,
	    valid_pct=0.2,
	    ds_tfms=get_transforms(do_flip=False),
	    size=224,
	    num_workers=4    
	)

	data.normalize(imagenet_stats)
view raw
import_split_transform.py hosted with ❤ by GitHub

Note that we split the data into training and validation set with an 80:20 ratio. The images are also resized to 224 pixels, which is usually a good value for image recognition tasks. Now that we have our data is imported, let’s look at some of the images:

data.show_batch(rows=3, figsize=(9,10))

A batch of images in the dataset

As we can see above, our dataset is rather ‘dirty’. Some of the images are not well-aligned and not properly cropped. But let’s quickly build the model and see how it performs.

Building the Model

We will be applying transfer learning and use ResNet-50 as our model. Run the following code to download the model along with the pre-trained weights:

learn = cnn_learner(data, models.resnet50, metrics=accuracy)

To train the layers, we can either use the fit or fit_one_cycle method. The fit method is the “normal” way of training a neural net with a constant learning rate. On the other hand, the fit_one_cycle method uses the ‘1 cycle policy’, which basically changes the learning rate over time to achieve better results.

#deep-learning #transfer-learning #machine-learning #art #data-science #deep learning

History of Computing PtI

We learn and know (hopefully) a basic history of the world, particularly major events like the French revolution, the American Civil War, World War I, World War II (wow lots of wars), the Spaceage etc. It is important to understand the concepts of these and many other historical events. Being able to recall the start year or the exact details of how such events unfolded is one thing, but on a human level, it is more important to understand the rationale, lessons and philosophy of major events. Ultimately history teaches us what makes us innately human. Furthermore, understanding history helps us realise the how and why we operate in today. History provides the context for today. It makes today seem ‘obvious,’ ‘justifable’ and ‘logical’ given the previous events that unfolded.

So, following this thread of logic, understanding the history of computers should help us understand how we have got to today. A today when computers moderate much of our communication with one another. A today where computers and screens are stared at for many (and often a majority) of our waking hours (especially during Covid). A today where the thought of working, socialising or learning without a computer would be an affront and a disadvantage. Just as major events like World War II and the Cold War have greatly contributed to today’s political and social climate. I would argue computers influence just as much (if not more) of our daily lives.

Therefore it is important for us to understand the evolution of computers to understand where we may be heading in our relationship with computers.

I would like to preface that the following articles outlining the history of computers by saying this is in no way an exhaustive history of the origin of computers. Some major events have been glossed over while other meaningful contributions omitted entirely.

Whilst the thought of history for some may make the eyes automatically glisten over, I will try and make the following series as painless and exciting as possible. While I paint a story of linear progress of computation, this is hindsight bias in action. We like to create a story of history attributing certain importance to some events and not others when in reality as these events were unfolding (and continue to unfold) it was not always obvious what was a gigantic discovery. It is only now with some distance that we can appreciate past event. This means perhaps in ten years this recount will emphasis other features and neglect some of the stories today we find so foundational to computer’s creation.

With all this in mind let’s begin !!

---

The first computers

Since their inception computers have taken over human work by performing tedious, complex and repetitive tasks. Interestingly, t_he__ word computer initially described humans_!! Initially computers were humans (often women) who were able to perform complex mathematical computations — usually with pen and paper. Often teams would work on the same calculation independently to confirm the end results. It is interesting to note that initally when electronic computers were developed they were referred to as such — electronic computers. With time as electronic computers became more and more pervasive and powerful, it became the human computer that was deemed obsolete and inefficient. The electronic was dropped and now when we discuss computers we think of nothing else besides our gracefull and versalite electronic tools. It is important to keep computer’s mathematical origin in mind as we will see it only further emphasises the never imagined pervasiveness and uses of computers today.

Our story begins with the humble abacus, generally considered the first computer. When researching I was puzzled how an abacus could be considered a computer. Luckily my curiosity was settled by a quick Google search (thank you Google). Google was even able to suggest my search before I completed typing ‘Why is the abacus considered the first computer’! I ended up on trusty Quora where one users: Chrissie Nysen put things simply-“Because it is used to compute things.” Though estimates vary, the abacus is thought to originate in Babylon approximately 5000 years ago. The role of the abacus was to ease in simple mathematical calculations- addition, subtraction, division and multiplication. In this sense, we can consider the abacus as a simple calculator. As farming, produce and populations increased in size, the abacus allowed the educated to more easily manage logistics. After the abacus the first computer, computer’s evolution remained dormant for some time……

#history #computer-science #history-of-technology #computers #computer-history #data science