Jack Downson

Jack Downson


Everything you need to know about TensorFlow 2.0

On June 26 of 2019, I will be giving a TensorFlow (TF) 2.0 workshop at the PAPIs.io LATAM conference in São Paulo. Aside from the happiness of being representing Daitan as the workshop host, I am very happy to talk about TF 2.0.

The idea of the workshop is to highlight what has changed from the previous 1.x version of TF. In this text, you can follow along with the main topics we are going to discuss. And of course, have a look at the Colab notebook for practical code.

Introduction to TensorFlow 2.0

TensorFlow is a general purpose high-performance computing library open sourced by Google in 2015. Since the beginning, its main focus was to provide high-performance APIs for building Neural Networks (NNs). However, with the advance of time and interest by the Machine Learning (ML) community, the lib has grown to a full ML ecosystem.

Currently, the library is experiencing its largest set of changes since its birth. TensorFlow 2.0 is currently in beta and brings many changes compared to TF 1.x. Let’s dive into the main ones.

Eager Execution By Default

To start, eager execution is the default way of running TF code.

As you might recall, to build a Neural Net in TF 1.x, we needed to define this abstract data structure called a Graph. Also, (as you probably have tried), if we attempted to print one of the graph nodes, we would not see the values we were expecting. Instead, we would see a reference to the graph node. To actually, run the graph, we needed to use an encapsulation called a Session. And using the Session.run() method, we could pass Python data to the graph and actually train our models.

With eager execution, this changes. Now, TensorFlow code can be run like normal Python code. Eagerly. Meaning that operations are created and evaluated at once.

TensorFlow 2.0 code looks a lot like NumPy code. In fact, TensorFlow and NumPy objects can easily be switched from one to the other. Hence, you do not need to worry about placeholders, Sessions, feed_dictionaties, etc.

API Cleanup

Many APIs like tf.gans, tf.app, tf.contrib, tf.flags are either gone or moved to separate repositories.

However, one of the most important cleanups relates to how we build models. You may remember that in TF 1.x we have many more than 1 or 2 different ways of building/training ML models.

Tf.slim, tf.layers, tf.contrib.layers, tf.keras are all possible APIs one can use to build NNs is TF 1.x. That not to include the Sequence to Sequence APIs in TF 1.x. And most of the time, it was not clear which one to choose for each situation.

Although many of these APIs have great features, they did not seem to converge to a common way of development. Moreover, if we trained a model in one of these APIs, it was not straight forward to reuse that code using the other ones.

In TF 2.0, tf.keras is the recommended high-level API.

As we will see, Keras API tries to address all possible use cases.

The Beginners API

From TF 1.x to 2.0, the beginner API did not change much. But now, Keras is the default and recommended high-level API. In summary, Keras is a set of layers that describes how to build neural networks using a clear standard. Basically, when we install TensorFlow using pip, we get the full Keras API plus some additional functionalities.

model = tf.keras.models.Sequential()
model.add(Dense(units=32, activation='relu'))
model.add(Dense(units=32, activation='relu'))
model.add(Dense(units=10, activation='softmax'))

# Configures the model for training.
# Define the model optimizer, the loss function and the accuracy metrics


The beginner’s API is called Sequential. It basically defines a neural network as a stack of layers. Besides its simplicity, it has some advantages. Note that we define our model in terms of a data structure (a stack of layers). As a result, it minimizes the probability of making errors due to model definition.


Keras-tuner is a dedicated library for hyper-parameter tuning of Keras models. As of this writing, the lib is in pre-alpha status but works fine on Colab with tf.keras and Tensorflow 2.0 beta.

It is a very simple concept. First, need to define a model building function that returns a compiled keras model. The function takes as input a parameter called hp. Using hp, we can define a range of candidate values that we can sample hyper-parameters values.

Below we build a simple model and optimize over 3 hyper-parameters. For the hidden units, we sample integer values between a pre-defined range. For dropout and learning rate, we choose at random, between some specified values.

def build_model(hp):
  # define the hyper parameter ranges for the learning rate, dropout and hidden unit
  hp_units = hp.Range('units', min_value=32, max_value=128, step=32)
  hp_lr = hp.Choice('learning_rate', values=[1e-2, 1e-3, 1e-4])
  hp_dropout = hp.Choice('dropout', values=[0.1,0.2,0.3])

  # build a Sequential model
  model = keras.Sequential()
  model.add(Dense(units=hp_units, activation='relu'))
  model.add(Dense(units=32, activation='relu'))
  model.add(layers.Dense(10, activation='softmax'))

  # compile and return the model
  return model
# create a Random Search tuner
tuner = RandomSearch(
    objective='val_accuracy', # define the metric to be optimized over
    directory='my_logs') # define the output log/checkpoints folder
 # start hyper-parameter optmization search
tuner.search(x_train, y_train,
             validation_data=(x_test, y_test))


Then, we create a tuner object. In this case, it implements a Random Search Policy. Lastly, we can start optimization using the search() method. It has the same signature as fit().

In the end, we can check the tuner summary results and choose the best model(s). Note that training logs and model checkpoints are all saved in the directory folder (my_logs). Also, the choice of minimizing or maximizing the objective (validation accuracy) is automatically infered.

Have a look at their Github page to learn more.

The Advanced API

The moment you see this type of implementation it goes back to Object Oriented programming. Here, your model is a Python class that extends tf.keras.Model. Model subclassing is an idea inspired by Chainer and relates very much to how PyTorch defines models.

With model Subclassing, we define the model layers in the class constructor. And the call() method handles the definition and execution of the forward pass.

class Model(tf.keras.Model):
  def __init__(self):
    # Define the layers here
    super(Model, self).__init__()
    self.conv1 = Conv2D(filters=8, kernel_size=4, padding="same", strides=1, input_shape=(IMAGE_HEIGHT,IMAGE_WIDTH,IMAGE_DEPTH))
    self.conv2 = Conv2D(filters=16, kernel_size=4, padding="same", strides=1)
    self.pool = MaxPool2D(pool_size=2, strides=2, padding="same")
    self.flat = Flatten()
    self.probs = Dense(units=N_CLASSES, activation='softmax', name="output")
  def call(self, x):
    # Define the forward pass
    net = self.conv1(x)
    net = self.pool(net)
    net = self.conv2(net)
    net = self.pool(net)
    net = self.flat(net)
    net = self.probs(net)
    return net
  def compute_output_shape(self, input_shape):
    # You need to override this function if you want to use the subclassed model
    # as part of a functional-style model.
    # Otherwise, this method is optional.
    shape = tf.TensorShape(input_shape).as_list()
    shape[-1] = self.num_classes
    return tf.TensorShape(shape)


Subclassing has many advantages. It is easier to perform a model inspection. We can, (using breakpoint debugging), stop at a given line and inspect the model’s activations or logits.

However, with great flexibility comes more bugs.

Model Subclassing requires more attention and knowledge from the programmer.

In general, your code is more prominent to errors (like model wiring).

Defining the Training Loop

The easiest way to train a model in TF 2.0 is by using the fit() method. fit() supports both types of models, Sequential and Subclassing. The only adjustment you need to do, if using model Subclassing, is to override the compute_output_shape() class method, otherwise, you can through it away. Other than that, you should be able to use fit() with either tf.data.Dataset or standard NumPy nd-arrays as input.

However, if you want a clear understanding of what is going on with the gradients or the loss, you can use the Gradient Tape. That is especially useful if you are doing research.

Using Gradient Tape, one can manually define each step of a training procedure. Each of the basic steps in training a neural net such as:

  • Forward pass
  • Loss function evaluation
  • Backward pass
  • Gradient descent step

is separately specified.

This is much more intuitive if one wants to get a feel of how a Neural Net is trained. If you want to check the loss values w.r.t the model weights or the gradient vectors itself, you can just print them out.

Gradient Tape gives much more flexibility. But just like Subclassing vs Sequential, more flexibility comes with an extra cost. Compared to the fit() method, here we need to define a training loop manually. As a natural consequence, it makes the code more prominent to bugs and harder to debug. I believe that is a great trade off that works ideally for code engineers (looking for standardized code), compared to researchers who usually are interested in developing something new.

Also, using fit() we can easily setup TensorBoard as we see next.

Setting up TensorBoard

You can easily setup an instance of TensorBoard using the fit() method. It also works on Jupyter/Colab notebooks.

In this case, you add TensorBoard as a callback to the fit method.

As long as you are using the fit() method, it works on both: Sequential and the Subclassing APIs.

# Load the TensorBoard notebook extension
%load_ext tensorboard

# create the tensorboard callback
tensorboard = TensorBoard(log_dir='logs/{}'.format(time.time()), histogram_freq=1)

# train the model
          validation_data=(x_test, y_test), 

# launch TensorBoard
%tensorboard --logdir logs


If you choose to use Model Subclassing and write the training loop yourself (using Grading Tape), you also need to define TensorBoard manually. It involves creating the summary files, using tf.summary.create_file_writer(), and specifying which variables you want to visualize.

As a worth noting point, there are many callbacks you can use. Some of the more useful ones are:

  • EarlyStopping: As the name implies, it sets up a rule to stop training when a monitored quantity has stopped improving.
  • ReduceLROnPlateau: Reduce the learning rate when a metric has stopped improving.
  • TerminateOnNaN: Callback that terminates training when a NaN loss is encountered.
  • LambdaCallback: Callback for creating simple, custom callbacks on-the-fly.

You can check the complete list at TensorFlow 2.0 callbacks.

Extracting Performance of your EagerCode

If you choose to train your model using Gradient Tape, you will notice a substantial decrease in performance.

Executing TF code eagerly is good for understanding, but it fails on performance. To avoid this problem, TF 2.0 introduces tf.function.

Basically, if you decorate a python function with tf.function, you are asking TensorFlow to take your function and convert it to a TF high-performance abstraction.

def train_step(images, labels):

  with tf.GradientTape() as tape:
    # forward pass
    predictions = model(images)
    # compute the loss
    loss = cross_entropy(tf.one_hot(labels, N_CLASSES), predictions)
  # get the gradients w.r.t the model's weights
  gradients = tape.gradient(loss, model.trainable_variables)
  # perform a gradient descent step
  optimizer.apply_gradients(zip(gradients, model.trainable_variables))
  # accumulates the training loss and accuracy
  train_accuracy(labels, predictions)


It means that the function will be marked for JIT compilation so that TensorFlow runs it as a graph. As a result, you get the performance benefits of TF 1.x (graphs) such as node pruning, kernel fusion, etc.

In short, the idea of TF 2.0 is that you can devise your code into smaller functions. Then, you can annotate the ones you wish using tf.function, to get this extra performance. It is best to decorate functions that represent the largest computing bottlenecks. These are usually the training loops or the model’s forward pass.

Note that when you decorate a function with tf.function, you loose some of the benefits of eager execution. In other words, you will not be able to setup breakpoints or use print() inside that section of code.

Save and Restore Models

Another great lack of standardization in TF 1.x is how we save/load trained models for production. TF 2.0 also tries to address this problem by defining a single API.

Instead of having many ways of saving models, TF 2.0 standardize to an abstraction called the SavedModel.

There is no much to say here. If you create a Sequential model or extend your class using tf.keras.Model, your class inherits from tf.train.Checkpoints. As a result, you can serialize your model to a SavedModel object.

# serialize your model to a SavedModel object
# It includes the entire graph, all variables and weights
model.save('/tmp/model', save_format='tf')

# load your saved model
model = tf.keras.models.load_model('/tmp/model')


SavedModels are integrated with the TensorFlow ecosystem. In other words, you will be able to deploy it to many different devices. These include mobile phones, edge devices, and servers.

Converting to TF-Lite

If you want to deploy a SavedModel to embedded devices like Raspberry Pi, Edge TPUs or your phone, use the TF Lite converter.

Note that in 2.0, the TFLiteConverter does not support frozen GraphDefs (usually generated in TF 1.x). If you want to convert a frozen GraphDefs to run in TF 2.0, you can use the tf.compat.v1.TFLiteConverter.

It is very common to perform post-training quantization before deploying to embedded devices. To do it with the TFLiteConverter, set the optimizations flag to “OPTIMIZE_FOR_SIZE”. This will quantize the model’s weights from floating point to 8-bits of precision. It will reduce the model size and improve latency with little degradation in model accuracy.

# create a TF Lite converter 
converter = tf.lite.TFLiteConverter.from_keras_model(model)

# performs model quantization to reduce the size of the model and improve latency
converter.optimizations = [tf.lite.Optimize.OPTIMIZE_FOR_SIZE]

tflite_model = converter.convert()


Note that this is an experimental flag, and it is subject to changes.

Converting to TensorFlow.js

To close up, we can also take the same SavedModel object and convert it to TensorFlow.js format. Then, we can load it using Javascript and run your model on the Browser.

!tensorflowjs_converter \
    --input_format=tf_saved_model \
    --saved_model_tags=serve \
    --output_format=tfjs_graph_model \
    /tmp/model \


First, you need to install TensorFlow.js via pip. Then, use the tensorflowjs_converter script to take your trained-model and convert to Javascript compatible code. Finally, you can load it and perform inference in Javascript.

You can also train models using Tesnorflow.js on the Browser.


To close off, I would like to mention some other capabilities of 2.0. First, we have seen that adding more layers to a Sequential or Subclassing model is very straightforward. And, although TF covers most of the popular layers like Conv2D, TransposeConv2D etc; you can always find yourself in a situation where you need something that is not available. That is especially true if you are reproducing some paper or doing research.

The good news is that we can develop our own Custom layers. Following the same Keras API, we can create a class and extend it to tf.keras.Layer. In fact, we can create custom activation functions, regularization layers, or metrics following a very similar pattern. Here is a good resource about it.

Also, we can convert existing TensorFlow 1.x code to TF 2.0. For this end, the TF team created the tf_upgrade_v2 utility.

This script does not convert TF 1.x code to 2.0 idiomatics. It basically uses tf.compat.v1 module for functions that got their namespaces changed. Also, if your legacy code uses tf.contrib, the script will not be able to convert it. You will probably need to use additional libraries or use the new TF 2.0 version of the missing functions.

Thanks for reading.

Understand TensorFlow by mimicking its API from scratchTheory

How to get started with Machine Learning in about 10 minutes

Introduction to Multilayer Neural Networks with TensorFlow’s Keras API

TensorFlow.js Crash Course – Machine Learning For The Web – Getting Started


What is GEEK

Buddha Community

Everything you need to know about TensorFlow 2.0
최 호민

최 호민


파이썬 코딩 무료 강의 - 이미지 처리, 얼굴 인식을 통한 캐릭터 씌우기를 해보아요

파이썬 코딩 무료 강의 (활용편6) - 이미지 처리, 얼굴 인식을 통한 캐릭터 씌우기를 해보아요

파이썬 무료 강의 (활용편6 - 이미지 처리)입니다.
OpenCV 를 이용한 다양한 이미지 처리 기법과 재미있는 프로젝트를 진행합니다.
누구나 볼 수 있도록 쉽고 재미있게 제작하였습니다. ^^

(0:00:00) 0.Intro
(0:00:31) 1.소개
(0:02:18) 2.활용편 6 이미지 처리 소개

[OpenCV 전반전]
(0:04:36) 3.환경설정
(0:08:41) 4.이미지 출력
(0:21:51) 5.동영상 출력 #1 파일
(0:29:58) 6.동영상 출력 #2 카메라
(0:34:23) 7.도형 그리기 #1 빈 스케치북
(0:39:49) 8.도형 그리기 #2 영역 색칠
(0:42:26) 9.도형 그리기 #3 직선
(0:51:23) 10.도형 그리기 #4 원
(0:55:09) 11.도형 그리기 #5 사각형
(0:58:32) 12.도형 그리기 #6 다각형
(1:09:23) 13.텍스트 #1 기본
(1:17:45) 14.텍스트 #2 한글 우회
(1:24:14) 15.파일 저장 #1 이미지
(1:29:27) 16.파일 저장 #2 동영상
(1:39:29) 17.크기 조정
(1:50:16) 18.이미지 자르기
(1:57:03) 19.이미지 대칭
(2:01:46) 20.이미지 회전
(2:06:07) 21.이미지 변형 - 흑백
(2:11:25) 22.이미지 변형 - 흐림
(2:18:03) 23.이미지 변형 - 원근 #1
(2:27:45) 24.이미지 변형 - 원근 #2

[반자동 문서 스캐너 프로젝트]
(2:32:50) 25.미니 프로젝트 1 - #1 마우스 이벤트 등록
(2:42:06) 26.미니 프로젝트 1 - #2 기본 코드 완성
(2:49:54) 27.미니 프로젝트 1 - #3 지점 선 긋기
(2:55:24) 28.미니 프로젝트 1 - #4 실시간 선 긋기

[OpenCV 후반전]
(3:01:52) 29.이미지 변형 - 이진화 #1 Trackbar
(3:14:37) 30.이미지 변형 - 이진화 #2 임계값
(3:20:26) 31.이미지 변형 - 이진화 #3 Adaptive Threshold
(3:28:34) 32.이미지 변형 - 이진화 #4 오츠 알고리즘
(3:32:22) 33.이미지 변환 - 팽창
(3:41:10) 34.이미지 변환 - 침식
(3:45:56) 35.이미지 변환 - 열림 & 닫힘
(3:54:10) 36.이미지 검출 - 경계선
(4:05:08) 37.이미지 검출 - 윤곽선 #1 기본
(4:15:26) 38.이미지 검출 - 윤곽선 #2 찾기 모드
(4:20:46) 39.이미지 검출 - 윤곽선 #3 면적

[카드 검출 & 분류기 프로젝트]
(4:27:42) 40.미니프로젝트 2

(4:31:57) 41.퀴즈

[얼굴인식 프로젝트]
(4:41:25) 42.환경설정 및 기본 코드 정리
(4:54:48) 43.눈과 코 인식하여 도형 그리기
(5:10:42) 44.그림판 이미지 씌우기
(5:20:52) 45.캐릭터 이미지 씌우기
(5:33:10) 46.보충설명
(5:40:53) 47.마치며 (학습 참고 자료)
(5:42:18) 48.Outro

수업에 필요한 이미지, 동영상 자료 링크입니다.

고양이 이미지 : https://pixabay.com/images/id-2083492/ 
크기 : 640 x 390  
파일명 : img.jpg

고양이 동영상 : https://www.pexels.com/video/7515833/ 
크기 : SD (360 x 640)  
파일명 : video.mp4

신문 이미지 : https://pixabay.com/images/id-350376/ 
크기 : 1280 x 853  
파일명 : newspaper.jpg

카드 이미지 1 : https://pixabay.com/images/id-682332/ 
크기 : 1280 x 1019  
파일명 : poker.jpg

책 이미지 : https://www.pexels.com/photo/1029807/ 
크기 : Small (640 x 853)  
파일명 : book.jpg

눈사람 이미지 : https://pixabay.com/images/id-1300089/ 
크기 : 1280 x 904  
파일명 : snowman.png

카드 이미지 2 : https://pixabay.com/images/id-161404/ 
크기 : 640 x 408  
파일명 : card.png

퀴즈용 동영상 : https://www.pexels.com/video/3121459/ 
크기 : HD (1280 x 720)  
파일명 : city.mp4

프로젝트용 동영상 : https://www.pexels.com/video/3256542/ 
크기 : Full HD (1920 x 1080)  
파일명 : face_video.mp4

프로젝트용 캐릭터 이미지 : https://www.freepik.com/free-vector/cute-animal-masks-video-chat-application-effect-filters-set_6380101.htm  
파일명 : right_eye.png (100 x 100), left_eye.png (100 x 100), nose.png (300 x 100)

무료 이미지 편집 도구 : https://pixlr.com/kr/
(Pixlr E -Advanced Editor)

#python #opencv 

Semantic Similarity Framework for Knowledge Graph


Sematch is an integrated framework for the development, evaluation, and application of semantic similarity for Knowledge Graphs (KGs). It is easy to use Sematch to compute semantic similarity scores of concepts, words and entities. Sematch focuses on specific knowledge-based semantic similarity metrics that rely on structural knowledge in taxonomy (e.g. depth, path length, least common subsumer), and statistical information contents (corpus-IC and graph-IC). Knowledge-based approaches differ from their counterpart corpus-based approaches relying on co-occurrence (e.g. Pointwise Mutual Information) or distributional similarity (Latent Semantic Analysis, Word2Vec, GLOVE and etc). Knowledge-based approaches are usually used for structural KGs, while corpus-based approaches are normally applied in textual corpora.

In text analysis applications, a common pipeline is adopted in using semantic similarity from concept level, to word and sentence level. For example, word similarity is first computed based on similarity scores of WordNet concepts, and sentence similarity is computed by composing word similarity scores. Finally, document similarity could be computed by identifying important sentences, e.g. TextRank.


KG based applications also meet similar pipeline in using semantic similarity, from concept similarity (e.g. http://dbpedia.org/class/yago/Actor109765278) to entity similarity (e.g. http://dbpedia.org/resource/Madrid). Furthermore, in computing document similarity, entities are extracted and document similarity is computed by composing entity similarity scores.


In KGs, concepts usually denote ontology classes while entities refer to ontology instances. Moreover, those concepts are usually constructed into hierarchical taxonomies, such as DBpedia ontology class, thus quantifying concept similarity in KG relies on similar semantic information (e.g. path length, depth, least common subsumer, information content) and semantic similarity metrics (e.g. Path, Wu & Palmer,Li, Resnik, Lin, Jiang & Conrad and WPath). In consequence, Sematch provides an integrated framework to develop and evaluate semantic similarity metrics for concepts, words, entities and their applications.

Getting started: 20 minutes to Sematch

Install Sematch

You need to install scientific computing libraries numpy and scipy first. An example of installing them with pip is shown below.

pip install numpy scipy

Depending on different OS, you can use different ways to install them. After sucessful installation of numpy and scipy, you can install sematch with following commands.

pip install sematch
python -m sematch.download

Alternatively, you can use the development version to clone and install Sematch with setuptools. We recommend you to update your pip and setuptools.

git clone https://github.com/gsi-upm/sematch.git
cd sematch
python setup.py install

We also provide a Sematch-Demo Server. You can use it for experimenting with main functionalities or take it as an example for using Sematch to develop applications. Please check our Documentation for more details.

Computing Word Similarity

The core module of Sematch is measuring semantic similarity between concepts that are represented as concept taxonomies. Word similarity is computed based on the maximum semantic similarity of WordNet concepts. You can use Sematch to compute multi-lingual word similarity based on WordNet with various of semantic similarity metrics.

from sematch.semantic.similarity import WordNetSimilarity
wns = WordNetSimilarity()

# Computing English word similarity using Li method
wns.word_similarity('dog', 'cat', 'li') # 0.449327301063
# Computing Spanish word similarity using Lin method
wns.monol_word_similarity('perro', 'gato', 'spa', 'lin') #0.876800984373
# Computing Chinese word similarity using  Wu & Palmer method
wns.monol_word_similarity('狗', '猫', 'cmn', 'wup') # 0.857142857143
# Computing Spanish and English word similarity using Resnik method
wns.crossl_word_similarity('perro', 'cat', 'spa', 'eng', 'res') #7.91166650904
# Computing Spanish and Chinese word similarity using Jiang & Conrad method
wns.crossl_word_similarity('perro', '猫', 'spa', 'cmn', 'jcn') #0.31023804699
# Computing Chinese and English word similarity using WPath method
wns.crossl_word_similarity('狗', 'cat', 'cmn', 'eng', 'wpath')#0.593666388463

Computing semantic similarity of YAGO concepts.

from sematch.semantic.similarity import YagoTypeSimilarity
sim = YagoTypeSimilarity()

#Measuring YAGO concept similarity through WordNet taxonomy and corpus based information content
sim.yago_similarity('http://dbpedia.org/class/yago/Dancer109989502','http://dbpedia.org/class/yago/Actor109765278', 'wpath') #0.642
sim.yago_similarity('http://dbpedia.org/class/yago/Dancer109989502','http://dbpedia.org/class/yago/Singer110599806', 'wpath') #0.544
#Measuring YAGO concept similarity based on graph-based IC
sim.yago_similarity('http://dbpedia.org/class/yago/Dancer109989502','http://dbpedia.org/class/yago/Actor109765278', 'wpath_graph') #0.423
sim.yago_similarity('http://dbpedia.org/class/yago/Dancer109989502','http://dbpedia.org/class/yago/Singer110599806', 'wpath_graph') #0.328

Computing semantic similarity of DBpedia concepts.

from sematch.semantic.graph import DBpediaDataTransform, Taxonomy
from sematch.semantic.similarity import ConceptSimilarity
concept = ConceptSimilarity(Taxonomy(DBpediaDataTransform()),'models/dbpedia_type_ic.txt')
concept.similarity('http://dbpedia.org/ontology/Actor','http://dbpedia.org/ontology/Film', 'path')
concept.similarity('http://dbpedia.org/ontology/Actor','http://dbpedia.org/ontology/Film', 'wup')
concept.similarity('http://dbpedia.org/ontology/Actor','http://dbpedia.org/ontology/Film', 'li')
concept.similarity('http://dbpedia.org/ontology/Actor','http://dbpedia.org/ontology/Film', 'res')
concept.similarity('http://dbpedia.org/ontology/Actor','http://dbpedia.org/ontology/Film', 'lin')
concept.similarity('http://dbpedia.org/ontology/Actor','http://dbpedia.org/ontology/Film', 'jcn')
concept.similarity('http://dbpedia.org/ontology/Actor','http://dbpedia.org/ontology/Film', 'wpath')

Computing semantic similarity of DBpedia entities.

from sematch.semantic.similarity import EntitySimilarity
sim = EntitySimilarity()
sim.similarity('http://dbpedia.org/resource/Madrid','http://dbpedia.org/resource/Barcelona') #0.409923677282

Evaluate semantic similarity metrics with word similarity datasets

from sematch.evaluation import WordSimEvaluation
from sematch.semantic.similarity import WordNetSimilarity
evaluation = WordSimEvaluation()
wns = WordNetSimilarity()
# define similarity metrics
wpath = lambda x, y: wns.word_similarity_wpath(x, y, 0.8)
# evaluate similarity metrics with SimLex dataset
evaluation.evaluate_metric('wpath', wpath, 'noun_simlex')
# performa Steiger's Z significance Test
evaluation.statistical_test('wpath', 'path', 'noun_simlex')
# define similarity metrics for Spanish words
wpath_es = lambda x, y: wns.monol_word_similarity(x, y, 'spa', 'path')
# define cross-lingual similarity metrics for English-Spanish
wpath_en_es = lambda x, y: wns.crossl_word_similarity(x, y, 'eng', 'spa', 'wpath')
# evaluate metrics in multilingual word similarity datasets
evaluation.evaluate_metric('wpath_es', wpath_es, 'rg65_spanish')
evaluation.evaluate_metric('wpath_en_es', wpath_en_es, 'rg65_EN-ES')

Evaluate semantic similarity metrics with category classification

Although the word similarity correlation measure is the standard way to evaluate the semantic similarity metrics, it relies on human judgements over word pairs which may not have same performance in real applications. Therefore, apart from word similarity evaluation, the Sematch evaluation framework also includes a simple aspect category classification. The task classifies noun concepts such as pasta, noodle, steak, tea into their ontological parent concept FOOD, DRINKS.

from sematch.evaluation import AspectEvaluation
from sematch.application import SimClassifier, SimSVMClassifier
from sematch.semantic.similarity import WordNetSimilarity

# create aspect classification evaluation
evaluation = AspectEvaluation()
# load the dataset
X, y = evaluation.load_dataset()
# define word similarity function
wns = WordNetSimilarity()
word_sim = lambda x, y: wns.word_similarity(x, y)
# Train and evaluate metrics with unsupervised classification model
simclassifier = SimClassifier.train(zip(X,y), word_sim)
evaluation.evaluate(X,y, simclassifier)

macro averge:  (0.65319812882333839, 0.7101245049198579, 0.66317566364913016, None)
micro average:  (0.79210167952791644, 0.79210167952791644, 0.79210167952791644, None)
weighted average:  (0.80842645056024054, 0.79210167952791644, 0.79639496616636352, None)
accuracy:  0.792101679528
             precision    recall  f1-score   support

    SERVICE       0.50      0.43      0.46       519
 RESTAURANT       0.81      0.66      0.73       228
       FOOD       0.95      0.87      0.91      2256
   LOCATION       0.26      0.67      0.37        54
   AMBIENCE       0.60      0.70      0.65       597
     DRINKS       0.81      0.93      0.87       752

avg / total       0.81      0.79      0.80      4406

Matching Entities with type using SPARQL queries

You can use Sematch to download a list of entities having a specific type using different languages. Sematch will generate SPARQL queries and execute them in DBpedia Sparql Endpoint.

from sematch.application import Matcher
matcher = Matcher()
# matching scientist entities from DBpedia
matcher.match_type('científico', 'spa')
matcher.match_type('科学家', 'cmn')
matcher.match_entity_type('movies with Tom Cruise')

Example of automatically generated SPARQL query.

SELECT DISTINCT ?s, ?label, ?abstract WHERE {
    ?s <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://dbpedia.org/class/yago/NuclearPhysicist110364643> . }
 UNION {  
    ?s <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://dbpedia.org/class/yago/Econometrician110043491> . }
 UNION {  
    ?s <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://dbpedia.org/class/yago/Sociologist110620758> . }
 UNION {  
    ?s <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://dbpedia.org/class/yago/Archeologist109804806> . }
 UNION {  
    ?s <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://dbpedia.org/class/yago/Neurolinguist110354053> . } 
    ?s <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://www.w3.org/2002/07/owl#Thing> . 
    ?s <http://www.w3.org/2000/01/rdf-schema#label> ?label . 
    FILTER( lang(?label) = "en") . 
    ?s <http://dbpedia.org/ontology/abstract> ?abstract . 
    FILTER( lang(?abstract) = "en") .
} LIMIT 5000

Entity feature extraction with Similarity Graph

Apart from semantic matching of entities from DBpedia, you can also use Sematch to extract features of entities and apply semantic similarity analysis using graph-based ranking algorithms. Given a list of objects (concepts, words, entities), Sematch compute their pairwise semantic similarity and generate similarity graph where nodes denote objects and edges denote similarity scores. An example of using similarity graph for extracting important words from an entity description.

from sematch.semantic.graph import SimGraph
from sematch.semantic.similarity import WordNetSimilarity
from sematch.nlp import Extraction, word_process
from sematch.semantic.sparql import EntityFeatures
from collections import Counter
tom = EntityFeatures().features('http://dbpedia.org/resource/Tom_Cruise')
words = Extraction().extract_nouns(tom['abstract'])
words = word_process(words)
wns = WordNetSimilarity()
word_graph = SimGraph(words, wns.word_similarity)
word_scores = word_graph.page_rank()
words, scores =zip(*Counter(word_scores).most_common(10))
print words
(u'picture', u'action', u'number', u'film', u'post', u'sport', 
u'program', u'men', u'performance', u'motion')


Ganggao Zhu, and Carlos A. Iglesias. "Computing Semantic Similarity of Concepts in Knowledge Graphs." IEEE Transactions on Knowledge and Data Engineering 29.1 (2017): 72-85.

Oscar Araque, Ganggao Zhu, Manuel Garcia-Amado and Carlos A. Iglesias Mining the Opinionated Web: Classification and Detection of Aspect Contexts for Aspect Based Sentiment Analysis, ICDM sentire, 2016.

Ganggao Zhu, and Carlos Angel Iglesias. "Sematch: Semantic Entity Search from Knowledge Graph." SumPre-HSWI@ ESWC. 2015.


You can post bug reports and feature requests in Github issues. Make sure to read our guidelines first. This project is still under active development approaching to its goals. The project is mainly maintained by Ganggao Zhu. You can contact him via gzhu [at] dit.upm.es

Why this name, Sematch and Logo?

The name of Sematch is composed based on Spanish "se" and English "match". It is also the abbreviation of semantic matching because semantic similarity metrics helps to determine semantic distance of concepts, words, entities, instead of exact matching.

The logo of Sematch is based on Chinese Yin and Yang which is written in I Ching. Somehow, it correlates to 0 and 1 in computer science.

Author: Gsi-upm
Source Code: https://github.com/gsi-upm/sematch 
License: View license

#python #jupyternotebook #graph 

Let Developers Just Need to Grasp only One Button Component

 From then on, developers only need to master one Button component, which is enough.

Support corners, borders, icons, special effects, loading mode, high-quality Neumorphism style.


✨ Features

Rich corner effect

Exquisite border decoration

Gradient effect

Flexible icon support

Intimate Loading mode

Cool interaction Special effects

More sense of space Shadow

High-quality Neumorphism style

🛠 Guide

⚙️ Parameters

🔩 Basic parameters

onPressedVoidCallbacktruenullClick callback. If null, FButton will enter an unavailable state
onPressedDownVoidCallbackfalsenullCallback when pressed
onPressedUpVoidCallbackfalsenullCallback when lifted
onPressedCancelVoidCallbackfalsenullCallback when cancel is pressed
styleTextStylefalsenulltext style
disableStyleTextStylefalsenullUnavailable text style
textStringfalsenullbutton text
colorColorfalsenullButton color
disabledColorColorfalsenullColor when FButton is unavailable
paddingEdgeInsetsGeometryfalsenullFButton internal spacing
cornerFCornerfalsenullConfigure corners of Widget
cornerStyleFCornerStylefalseFCornerStyle.roundConfigure the corner style of Widget. round-rounded corners, bevel-beveled
strokeColorColorfalseColors.blackBorder color
strokeWidthdoublefalse0Border width. The border will appear when strokeWidth > 0
gradientGradientfalsenullConfigure gradient colors. Will override the color
activeMaskColorColorColors.transparentThe color of the mask when pressed
surfaceStyleFSurfacefalseFSurface.FlatSurface style. Default [FSurface.Flat]. See [FSurface] for details

💫 Effect parameters

clickEffectboolfalsefalseWhether to enable click effects
hoverColorColorfalsenullFButton color when hovering
onHoverValueChangedfalsenullCallback when the mouse enters/exits the component range
highlightColorColorfalsenullThe color of the FButton when touched. effect:true required

🔳 Shadow parameters

shadowColorColorfalseColors.greyShadow color
shadowOffsetOffsetfalseOffset.zeroShadow offset
shadowBlurdoublefalse1.0Shadow blur degree, the larger the value, the larger the shadow range

🖼 Icon & Loading parameters

imageWidgetfalsenullAn icon can be configured for FButton
imageMargindoublefalse6.0Spacing between icon and text
imageAlignmentImageAlignmentfalseImageAlignment.leftRelative position of icon and text
loadingboolfalsefalseWhether to enter the Loading state
loadingWidgetWidgetfalsenullLoading widget in loading state. Will override the default Loading effect
clickLoadingboolfalsefalseWhether to enter Loading state after clicking FButton
loadingColorColorfalsenullLoading colors
loadingStrokeWidthdoublefalse4.0Loading width
hideTextOnLoadingboolfalsefalseWhether to hide text in the loading state
loadingTextStringfalsenullLoading text
loadingSizedoublefalse12Loading size

🍭 Neumorphism Style

isSupportNeumorphismboolfalsefalseWhether to support the Neumorphism style. Open this item [highlightColor] will be invalid
lightOrientationFLightOrientationfalseFLightOrientation.LeftTopValid when [isSupportNeumorphism] is true. The direction of the light source is divided into four directions: upper left, lower left, upper right, and lower right. Used to control the illumination direction of the light source, which will affect the highlight direction and shadow direction
highlightShadowColorColorfalsenullAfter the Neumorphism style is turned on, the bright shadow color

📺 Demo

🔩 Basic Demo

// FButton #1
  height: 40,
  alignment: Alignment.center,
  text: "FButton #1",
  style: TextStyle(color: Colors.white),
  color: Color(0xffffab91),
  onPressed: () {},

// FButton #2
  padding: const EdgeInsets.fromLTRB(12, 8, 12, 8),
  text: "FButton #2",
  style: TextStyle(color: Colors.white),
  color: Color(0xffffab91),
  corner: FCorner.all(6.0),

// FButton #3
  padding: const EdgeInsets.fromLTRB(12, 8, 12, 8),
  text: "FButton #3",
  style: TextStyle(color: Colors.white),
  disableStyle: TextStyle(color: Colors.black38),
  color: Color(0xffF8AD36),

  /// set disable Color
  disabledColor: Colors.grey[300],
  corner: FCorner.all(6.0),

By simply configuring text andonPressed, you can construct an available FButton.

If onPressed is not set, FButton will be automatically recognized as not unavailable. At this time, ** FButton ** will have a default unavailable status style.

You can also freely configure the style of FButton when it is not available via the disabledXXX attribute.

🎈 Corner & Stroke

// #1
  width: 130,
  text: "FButton #1",
  style: TextStyle(color: Colors.white),
  color: Color(0xffFF7043),
  onPressed: () {},
  clickEffect: true,
  /// 配置边角大小
  /// set corner size
  corner: FCorner.all(25),

// #2
  width: 130,
  text: "FButton #2",
  style: TextStyle(color: Colors.white),
  color: Color(0xffFFA726),
  onPressed: () {},
  clickEffect: true,
  corner: FCorner(
    leftBottomCorner: 40,
    leftTopCorner: 6,
    rightTopCorner: 40,
    rightBottomCorner: 6,

// #3
  width: 130,
  text: "FButton #3",
  style: TextStyle(color: Colors.white),
  color: Color(0xffFFc900),
  onPressed: () {},
  clickEffect: true,
  corner: FCorner(leftTopCorner: 10),
  /// 设置边角风格
  /// set corner style
  cornerStyle: FCornerStyle.bevel,
  strokeWidth: 0.5,
  strokeColor: Color(0xffF9A825),

// #4
  width: 130,
  padding: EdgeInsets.fromLTRB(6, 16, 30, 16),
  text: "FButton #4",
  style: TextStyle(color: Colors.white),
  color: Color(0xff00B0FF),
  onPressed: () {},
  clickEffect: true,
  corner: FCorner(
      rightTopCorner: 25,
      rightBottomCorner: 25),
  cornerStyle: FCornerStyle.bevel,
  strokeWidth: 0.5,
  strokeColor: Color(0xff000000),

You can add rounded corners to FButton via the corner property. You can even control each fillet individually。

By default, the corners of FButton are rounded. By setting cornerStyle: FCornerStyle.bevel, you can get a bevel effect.

FButton supports control borders, provided that strokeWidth> 0 can get the effect 🥳.

🌈 Gradient

  width: 100,
  height: 60,
  text: "#1",
  style: TextStyle(color: Colors.white),
  color: Color(0xffFFc900),
  /// 配置渐变色
  /// set gradient
  gradient: LinearGradient(colors: [
  onPressed: () {},
  clickEffect: true,
  corner: FCorner.all(8),

Through the gradient attribute, you can build FButton with gradient colors. You can freely build many types of gradient colors.

🍭 Icon

  width: 88,
  height: 38,
  padding: EdgeInsets.all(0),
  text: "Back",
  style: TextStyle(color: Colors.white),
  color: Color(0xffffc900),
  onPressed: () {
    toast(context, "Back!");
  clickEffect: true,
  corner: FCorner(
    leftTopCorner: 25,
    leftBottomCorner: 25,),
  /// 配置图标
  /// set icon
  image: Icon(
    color: Colors.white,
    size: 12,

  /// 配置图标与文字的间距
  /// Configure the spacing between icon and text
  imageMargin: 8,

  onPressed: () {},
  image: Icon(
    color: Colors.grey,
  imageMargin: 8,

  /// 配置图标与文字相对位置
  /// Configure the relative position of icons and text
  imageAlignment: ImageAlignment.top,
  text: "Print",
  style: TextStyle(color: textColor),
  color: Colors.transparent,

The image property can set an image for FButton and you can adjust the position of the image relative to the text, throughimageAlignment.

If the button does not need a background, just set color: Colors.transparent.

🔥 Effect

  width: 200,
  text: "Try Me!",
  style: TextStyle(color: textColor),
  color: Color(0xffffc900),
  onPressed: () {},
  clickEffect: true,
  corner: FCorner.all(9),
  /// 配置按下时颜色
  /// set pressed color
  highlightColor: Color(0xffE65100).withOpacity(0.20),
  /// 配置 hover 状态时颜色
  /// set hover color
  hoverColor: Colors.redAccent.withOpacity(0.16),

The highlight color of FButton can be configured through the highlightColor property。

hoverColor can configure the color when the mouse moves to the range of FButton, which will be used during Web development.

🔆 Loading

  text: "Click top loading",
  style: TextStyle(color: textColor),
  color: Color(0xffffc900),

  /// 配置 loading 大小
  /// set loading size
  loadingSize: 15,

  /// 配置 loading 与文本的间距
  // Configure the spacing between loading and text
  imageMargin: 6,
  /// 配置 loading 的宽
  /// set loading width
  loadingStrokeWidth: 2,

  /// 是否支持点击自动开始 loading
  /// Whether to support automatic loading by clicking
  clickLoading: true,

  /// 配置 loading 的颜色
  /// set loading color
  loadingColor: Colors.white,

  /// 配置 loading 状态时的文本
  /// set loading text
  loadingText: "Loading...",

  /// 配置 loading 与文本的相对位置
  /// Configure the relative position of loading and text
  imageAlignment: ImageAlignment.top,

// #2
  width: 170,
  height: 70,
  text: "Click to loading",
  style: TextStyle(color: textColor),
  color: Color(0xffffc900),
  onPressed: () { },
  imageMargin: 8,
  loadingSize: 15,
  loadingStrokeWidth: 2,
  clickLoading: true,
  loadingColor: Colors.white,
  loadingText: "Loading...",

  /// loading 时隐藏文本
  /// Hide text when loading
  hideTextOnLoading: true,

  width: 170,
  height: 70,
  alignment: Alignment.center,
  text: "Click to loading",
  style: TextStyle(color: Colors.white),
  color: Color(0xff90caf9),
  imageMargin: 8,
  clickLoading: true,
  hideTextOnLoading: true,

  /// 配置自定义 loading 样式
  /// Configure custom loading style
  loadingWidget: CupertinoActivityIndicator(),

Through the loading attribute, you can configure Loading effects for ** FButton **.

When FButton is in Loading state, FButton will enter an unavailable state, onPress will no longer be triggered, and unavailable styles will also be applied.

At the same time loadingText will overwritetext if it is not null.

The click start Loading effect can be achieved through the clickLoading attribute.

The position of loading will be affected by theimageAlignment attribute.

When hideTextOnLoading: true, if FButton is inloading state, its text will be hidden.

Through loadingWidget, developers can set completely customized loading styles.


  width: 200,
  text: "Shadow",
  textColor: Colors.white,
  color: Color(0xffffc900),
  onPressed: () {},
  clickEffect: true,
  corner: FCorner.all(28),
  /// 配置阴影颜色
  /// set shadow color
  shadowColor: Colors.black87,

  /// 设置组件高斯与阴影形状卷积的标准偏差。
  /// Sets the standard deviation of the component's Gaussian convolution with the shadow shape.
  shadowBlur: _shadowBlur,

FButton allows you to configure the color, size, and position of the shadow.

🍭 Neumorphism Style


  /// 开启 Neumorphism 支持
  /// Turn on Neumorphism support
  isSupportNeumorphism: true,

  /// 配置光源方向
  /// Configure light source direction
  lightOrientation: lightOrientation,

  /// 配置亮部阴影
  /// Configure highlight shadow
  highlightShadowColor: Colors.white,

  /// 配置暗部阴影
  /// Configure dark shadows
  shadowColor: mainShadowColor,
  strokeColor: mainBackgroundColor,
  strokeWidth: 3.0,
  width: 190,
  height: 60,
  text: "FWidget",
  style: TextStyle(
      color: mainTextTitleColor, fontSize: neumorphismSize_2_2),
  alignment: Alignment.center,
  color: mainBackgroundColor,

FButton brings an incredible, ultra-high texture Neumorphism style to developers.

Developers only need to configure the isSupportNeumorphism parameter to enable and disable the Neumorphism style.

If you want to adjust the style of Neumorphism, you can make subtle adjustments through several attributes related to Shadow, among which:

shadowColor: configure the shadow of the shadow

highlightShadowColor: configure highlight shadow

FButton also provides lightOrientation parameters, and even allows developers to adjust the care angle, and has obtained different Neumorphism effects.

😃 How to use?

Add dependencies in the project pubspec.yaml file:

🌐 pub dependency

  fbutton: ^<version number>

⚠️ Attention,please go to [pub] (https://pub.dev/packages/fbutton) to get the latest version number of FButton

🖥 git dependencies

      url: 'git@github.com:Fliggy-Mobile/fbutton.git'
      ref: '<Branch number or tag number>'

Use this package as a library

Depend on it

Run this command:

With Flutter:

 $ flutter pub add fbutton_nullsafety

This will add a line like this to your package's pubspec.yaml (and run an implicit flutter pub get):

  fbutton_nullsafety: ^5.0.0

Alternatively, your editor might support or flutter pub get. Check the docs for your editor to learn more.

Import it

Now in your Dart code, you can use:

import 'package:fbutton_nullsafety/fbutton_nullsafety.dart';

Download Details:

Author: Fliggy-Mobile

Source Code: https://github.com/Fliggy-Mobile/fbutton

#button  #flutter 

伊藤  直子

伊藤 直子


【 初心者向け】C言語でのマルチスレッド の概要

ニューヨークで働き、ウォール街中のプログラマーと話をしていると、ほとんどのリアルタイムプログラミングアプリケーションで期待される共通の知識の糸に気づきました。その知識はマルチスレッドとして知られています。私はプログラミングの世界を移動し、潜在的なプログラミング候補者にインタビューを行ったので、マルチスレッドについてほとんど知られていないことや、スレッドが適用される理由や方法に驚かされることは決してありません。Vance Morrisonによって書かれた一連の優れた記事で、MSDNはこの問題に対処しようとしました:(MSDNの8月号、すべての開発者がマルチスレッドアプリについて知っておくべきこと、および10月号はマルチスレッドでのローロック技術の影響を理解するを参照してください)。アプリ











初心者のプログラマーが最初にスレッド化を学ぶとき、彼らはプログラムでスレッド化を使用する可能性に魅了される可能性があります。彼らは実際にスレッドハッピーになるかもしれません  詳しく説明させてください、

1日目)プログラマーは、スレッドを生成できることを学び、プログラムで1つの新しいスレッドCool!の作成を開始します 。







おなじみですか?マルチスレッドプログラムを初めて設計しようとしたほとんどの人は、スレッドの設計知識が豊富であっても、おそらくこれらの毎日の箇条書きの少なくとも1つまたは2つを経験したことがあります。スレッド化が悪いことだとほのめかしているわけではありません。プログラムでスレッド化の効率を上げるプロセスでは、非常に注意してください。  シングルスレッドプログラムとは異なり、同時に多くのプロセスを処理しているため、複数の従属変数を持つ複数のプロセスを追跡するのは非常に難しい場合があります。ジャグリングと同じようにマルチスレッドを考えてください。手で1つのボールをジャグリングするのは(退屈ではありますが)かなり簡単です。ただし、これらのボールのうち2つを空中に置くように挑戦された場合、その作業は少し難しくなります。3、4、および5の場合、ボールは次第に難しくなります。ボールの数が増えると、実際にボールを落とす可能性が高くなります。 一度にたくさんのボールをジャグリングするには、知識、スキル、正確なタイミングが必要です。マルチスレッドもそうです。 









// shared memory variable between the two threads  
// used to indicate which thread we are in  
private string _threadOutput = "";  
/// <summary>  
/// Thread 1: Loop continuously,  
/// Thread 1: Displays that we are in thread 1  
/// </summary>  
void DisplayThread1()  
      while (_stopThreads == false)  
            Console.WriteLine("Display Thread 1");  
            // Assign the shared memory to a message about thread #1  
            _threadOutput = "Hello Thread1";  
            Thread.Sleep(1000);  // simulate a lot of processing   
            // tell the user what thread we are in thread #1, and display shared memory  
            Console.WriteLine("Thread 1 Output --> {0}", _threadOutput);  

/// <summary>  
/// Thread 2: Loop continuously,  
/// Thread 2: Displays that we are in thread 2  
/// </summary>  
void DisplayThread2()  
      while (_stopThreads == false)  
        Console.WriteLine("Display Thread 2");  
       // Assign the shared memory to a message about thread #2  
        _threadOutput = "Hello Thread2";  
        Thread.Sleep(1000);  // simulate a lot of processing  
       // tell the user we are in thread #2  
        Console.WriteLine("Thread 2 Output --> {0}", _threadOutput);  
      // construct two threads for our demonstration;  
      Thread thread1 = new Thread(new ThreadStart(DisplayThread1));  
      Thread thread2 = new Thread(new ThreadStart(DisplayThread2));  
      // start them  






スレッド2の出力->ハロースレッド1とスレッド1の出力->ハロースレッド2が表示されることがあります。スレッドの出力がコードと一致しません。コードを見て、それを目で追っていますが、_threadOutput = "Hello Thread 2"、Sleep、Write "Thread 2-> Hello Thread 2"ですが、このシーケンスで必ずしも最終結果が得られるとは限りません。 


このようなマルチスレッドプログラムでは、理論的にはコードが2つのメソッドDisplayThread1とDisplayThread2を同時に実行しているためです。各メソッドは変数_threadOutputを共有します。したがって、_threadOutputにはスレッド#1で値 "Hello Thread1"が割り当てられ、2行後にコンソールに_threadOutputが表示されますが、スレッド#1がそれを割り当てて表示する時間の間のどこかで、スレッド#2が_threadOutputを割り当てる可能性があります。値「HelloThread2」。これらの奇妙な結果が発生する可能性があるだけでなく、図2に示す出力に見られるように、非常に頻繁に発生します。この痛みを伴うスレッドの問題は、競合状態として知られるスレッドプログラミングで非常に一般的なバグです。 この例は、よく知られているスレッドの問題の非常に単純な例です。この問題は、参照されている変数やスレッドセーフでない変数を指すコレクションなどを介して、プログラマーからはるかに間接的に隠されている可能性があります。図2では症状は露骨ですが、競合状態は非常にまれにしか現れず、1分に1回、1時間に1回、または3日後に断続的に現れる可能性があります。レースは、その頻度が低く、再現が非常に難しいため、おそらくプログラマーにとって最悪の悪夢です。


競合状態を回避する最善の方法は、スレッドセーフなコードを作成することです。コードがスレッドセーフである場合、いくつかの厄介なスレッドの問題が発生するのを防ぐことができます。スレッドセーフなコードを書くためのいくつかの防御策があります。1つは、メモリの共有をできるだけ少なくすることです。クラスのインスタンスを作成し、それが1つのスレッドで実行され、次に同じクラスの別のインスタンスを作成し、それが別のスレッドで実行される場合、静的変数が含まれていない限り、クラスはスレッドセーフです。 。2つのクラスはそれぞれ、独自のフィールド用に独自のメモリを作成するため、共有メモリはありません。クラスに静的変数がある場合、またはクラスのインスタンスが他の複数のスレッドによって共有されている場合は、他のクラスがその変数の使用を完了するまで、一方のスレッドがその変数のメモリを使用できないようにする方法を見つける必要があります。ロック。  C#を使用すると、Monitorクラスまたはlock {}構造のいずれかを使用してコードをロックできます。(lock構造は、実際にはtry-finallyブロックを介してMonitorクラスを内部的に実装しますが、プログラマーからこれらの詳細を隠します)。リスト1の例では、共有_threadOutput変数を設定した時点から、コンソールへの実際の出力まで、コードのセクションをロックできます。コードのクリティカルセクションを両方のスレッドでロックして、どちらか一方に競合が発生しないようにします。メソッド内をロックする最も速くて汚い方法は、このポインターをロックすることです。このポインタをロックすると、クラスインスタンス全体がロックされるため、ロック内でクラスのフィールドを変更しようとするスレッドはすべてブロックされます。。ブロッキングとは、変数を変更しようとしているスレッドが、ロックされたスレッドでロックが解除されるまで待機することを意味します。スレッドは、lock {}構造の最後のブラケットに到達すると、ロックから解放されます。


/// <summary>  
/// Thread 1, Displays that we are in thread 1 (locked)  
 /// </summary>  
 void DisplayThread1()  
       while (_stopThreads == false)  
          // lock on the current instance of the class for thread #1  
             lock (this)  
                   Console.WriteLine("Display Thread 1");  
                   _threadOutput = "Hello Thread1";  
                   Thread.Sleep(1000);  // simulate a lot of processing  
                   // tell the user what thread we are in thread #1  
                   Console.WriteLine("Thread 1 Output --> {0}", _threadOutput);  
             }// lock released  for thread #1 here  

/// <summary>  
/// Thread 1, Displays that we are in thread 1 (locked)  
 /// </summary>  
 void DisplayThread2()  
       while (_stopThreads == false)  
           // lock on the current instance of the class for thread #2  
             lock (this)  
                   Console.WriteLine("Display Thread 2");  
                   _threadOutput = "Hello Thread2";  
                   Thread.Sleep(1000);  // simulate a lot of processing  
                   // tell the user what thread we are in thread #1  
                   Console.WriteLine("Thread 2 Output --> {0}", _threadOutput);  
             } // lock released  for thread #2 here  





.NETは、スレッドの制御に役立つ多くのメカニズムを提供します。別のスレッドが共有メモリの一部を処理している間、スレッドをブロックしたままにする別の方法は、AutoResetEventを使用することです。AutoResetEventクラスには、SetとWaitOneの2つのメソッドがあります。これらの2つの方法は、スレッドのブロックを制御するために一緒に使用できます。AutoResetEventがfalseで初期化されると、プログラムは、AutoResetEventでSetメソッドが呼び出されるまで、WaitOneを呼び出すコード行で停止します。AutoResetEventでSetメソッドが実行されると、スレッドのブロックが解除され、WaitOneを超えて続行できるようになります。次回WaitOneが呼び出されると、自動的にリセットされるため、プログラムは、WaitOneメソッドが実行されているコード行で再び待機(ブロック)します。この「停止とトリガー」を使用できます Setを呼び出して、別のスレッドがブロックされたスレッドを解放する準備ができるまで、あるスレッドをブロックするメカニズム。リスト3は、AutoResetEventを使用して、ブロックされたスレッドが待機し、ブロックされていないスレッドが実行されてコンソールに_threadOutputを表示している間に、互いにブロックする同じ2つのスレッドを示しています。最初に、_blockThread1はfalseを通知するように初期化され、_blockThread2はtrueを通知するように初期化されます。これは、_blockThread2がDisplayThread_2のループを最初に通過するときに、WaitOne呼び出しを続行できるようになる一方で、_blockThread1はDisplayThread_1のWaitOne呼び出しをブロックすることを意味します。_blockThread2がスレッド2のループの終わりに達すると、スレッド1をブロックから解放するためにSetを呼び出して_blockThread1に信号を送ります。次に、スレッド2は、スレッド1がループの終わりに到達して_blockThread2でSetを呼び出すまで、WaitOne呼び出しで待機します。スレッド1で呼び出されたセットはスレッド2のブロックを解放し、プロセスが再開されます。両方のAutoResetEvents(_blockThread1と_blockThread2)を最初にfalseを通知するように設定した場合、両方のスレッドが互いにトリガーする機会なしにループの進行を待機し、デッドロック。 


AutoResetEvent _blockThread1 = new AutoResetEvent(false);  
AutoResetEvent _blockThread2 = new AutoResetEvent(true);  
/// <summary>  
/// Thread 1, Displays that we are in thread 1  
/// </summary>  
void DisplayThread_1()  
      while (_stopThreads == false)  
               // block thread 1  while the thread 2 is executing  
                // Set was called to free the block on thread 1, continue executing the code  
                  Console.WriteLine("Display Thread 1");  
                  _threadOutput = "Hello Thread 1";  
                  Thread.Sleep(1000);  // simulate a lot of processing  
                   // tell the user what thread we are in thread #1  
                  Console.WriteLine("Thread 1 Output --> {0}", _threadOutput);  
                // finished executing the code in thread 1, so unblock thread 2  
/// <summary>  
/// Thread 2, Displays that we are in thread 2  
/// </summary>  
void DisplayThread_2()  
      while (_stopThreads == false)  
            // block thread 2  while thread 1 is executing  
            // Set was called to free the block on thread 2, continue executing the code  
                  Console.WriteLine("Display Thread 2");  
                  _threadOutput = "Hello Thread 2";  
                  Thread.Sleep(1000);  // simulate a lot of processing  
                   // tell the user we are in thread #2  
                  Console.WriteLine("Thread 2 Output --> {0}", _threadOutput);   
            // finished executing the code in thread 2, so unblock thread 1  







Were  Joyce

Were Joyce


Spring Integration AWS 2.5.1 and Spring Cloud Stream Kinesis Binder 2.2.0 Available

Dear Spring Community,

Today it’s my pleasure to announce releases of Spring Integration for Amazon Web Services extension versions 2.5.1 and Spring Cloud Stream Binder for AWS Kinesis 2.2.0.

These releases can be downloaded from Maven Central:

compile 'org.springframework.integration:spring-integration-aws:2.5.1'

If you don’t use Kinesis Binder. Or via Binder dependency:

compile 'org.springframework.cloud:spring-cloud-stream-binder-kinesis:2.2.0'

Spring Integration for AWS extension of 2.5.1 has some minor bug fixes after the previous 2.5.0 announcement.

The Spring Cloud Stream Binder for AWS Kinesis 2.2.0 is the first release in the project generation which is based on a new independent Spring Cloud for AWS artifact. It also was upgraded to the latest Spring Cloud 2020.0.3 and has some other bug fixes and improvements.

#[object object] #aws 2.5.1 #spring cloud #spring cloud stream kinesis binder 2.2.0 available #spring integration aws 2.5.1 and spring cloud stream kinesis binder 2.2.0 available