Learn Python Programming - Sets in Python

Sets are a key way to store data in Python. In this tutorial, we show you how to create sets, how to add and remove elements from a set, and how to take the union and intersection of two sets. We will talk about lists, dictionaries and tuples in separate videos.

A Set is an unordered collection data type that is iterable, mutable and has no duplicate elements. Python’s set class represents the mathematical notion of a set. The major advantage of using a set, as opposed to a list, is that it has a highly optimized method for checking whether a specific element is contained in the set. This is based on a data structure known as a hash table. Since sets are unordered, we cannot access items using indexes like we do in lists.

# Python program to 
# demonstrate sets 

# Same as {"a", "b", "c"} 
myset = set(["a", "b", "c"]) 
print(myset) 

# Adding element to the set 
myset.add("d") 
print(myset)

Output:

{'c', 'b', 'a'}
{'d', 'c', 'b', 'a'}

Frozen Sets

Frozen sets in Python are immutable objects that only support methods and operators that produce a result without affecting the frozen set or sets to which they are applied. While elements of a set can be modified at any time, elements of the frozen set remain the same after creation.
If no parameters are passed, it returns an empty frozenset.

# Python program to demonstrate differences 
# between normal and frozen set 

# Same as {"a", "b","c"} 
normal_set = set(["a", "b","c"]) 

print("Normal Set") 
print(normal_set) 

# A frozen set 
frozen_set = frozenset(["e", "f", "g"]) 

print("\nFrozen Set") 
print(frozen_set) 

# Uncommenting below line would cause error as 
# we are trying to add element to a frozen set 
# frozen_set.add("h")

Output:

Normal Set
set(['a', 'c', 'b'])

Frozen Set
frozenset(['e', 'g', 'f'])

Internal working of Set

This is based on a data structure known as a hash table.
If Multiple values are present at the same index position, then the value is appended to that index position, to form a Linked List. In, Python Sets are implemented using dictionary with dummy variables, where key beings the members set with greater optimizations to the time complexity.

Set Implementation:-

Sets with Numerous operations on a single HashTable:-

Methods for Sets

Adding elements

Insertion in set is done through set.add() function, where an appropriate record value is created to store in the hash table. Same as checking for an item, i.e., O(1) on average. However, in worst case it can become O(n).

# A Python program to 
# demonstrate adding elements 
# in a set 

# Creating a Set 
people = {"Jay", "Idrish", "Archi"} 

print("People:", end = " ") 
print(people) 

# This will add Daxit 
# in the set 
people.add("Daxit") 

# Adding elements to the 
# set using iterator 
for i in range(1, 6): 
	people.add(i) 

print("\nSet after adding element:", end = " ") 
print(people)

Output:

People: {'Idrish', 'Archi', 'Jay'}

Set after adding element: {1, 2, 3, 4, 5, 'Idrish', 'Archi', 'Jay', 'Daxit'}

Union

Two sets can be merged using union() function or | operator. Both Hash Table values are accessed and traversed with merge operation perform on them to combine the elements, at the same time duplicates are removed. Time Complexity of this is O(len(s1) + len(s2)) where s1 and s2 are two sets whose union needs to be done.

# Python Program to 
# demonstrate union of 
# two sets 

people = {"Jay", "Idrish", "Archil"} 
vampires = {"Karan", "Arjun"} 
dracula = {"Deepanshu", "Raju"} 

# Union using union() 
# function 
population = people.union(vampires) 

print("Union using union() function") 
print(population) 

# Union using "|" 
# operator 
population = people|dracula 

print("\nUnion using '|' operator") 
print(population)

Output:

Union using union() function
{'Karan', 'Idrish', 'Jay', 'Arjun', 'Archil'}

Union using '|' operator
{'Deepanshu', 'Idrish', 'Jay', 'Raju', 'Archil'}

Intersection

This can be done through intersection() or & operator. Common Elements are selected. They are similar to iteration over the Hash lists and combining the same values on both the Table. Time Complexity of this is O(min(len(s1), len(s2)) where s1 and s2 are two sets whose union needs to be done.

# Python program to 
# demonstrate intersection 
# of two sets 

set1 = set() 
set2 = set() 

for i in range(5): 
	set1.add(i) 

for i in range(3,9): 
	set2.add(i) 

# Intersection using 
# intersection() function 
set3 = set1.intersection(set2) 

print("Intersection using intersection() function") 
print(set3) 

# Intersection using 
# "&" operator 
set3 = set1 & set2 

print("\nIntersection using '&' operator") 
print(set3)

Output:

Intersection using intersection() function
{3, 4}

Intersection using '&' operator
{3, 4}

Difference

To find difference in between sets. Similar to find difference in linked list. This is done through difference() or – operator. Time complexity of finding difference s1 – s2 is O(len(s1))

# Python program to 
# demonstrate difference 
# of two sets 

set1 = set() 
set2 = set() 

for i in range(5): 
	set1.add(i) 

for i in range(3,9): 
	set2.add(i) 

# Difference of two sets 
# using difference() function 
set3 = set1.difference(set2) 

print(" Difference of two sets using difference() function") 
print(set3) 

# Difference of two sets 
# using '-' operator 
set3 = set1 - set2 

print("\nDifference of two sets using '-' operator") 
print(set3)

Output:

Difference of two sets using difference() function
{0, 1, 2}

Difference of two sets using '-' operator
{0, 1, 2}

Clearing sets

Clear() method empties the whole set.

# Python program to 
# demonstrate clearing 
# of set 

set1 = {1,2,3,4,5,6} 

print("Initial set") 
print(set1) 

# This method will remove 
# all the elements of the set 
set1.clear() 

print("\nSet after using clear() function") 
print(set1)

Output:

Initial set
{1, 2, 3, 4, 5, 6}

Set after using clear() function
set()

However, there are two major pitfalls in Python sets:

The set doesn’t maintain elements in any particular order.
Only instances of immutable types can be added to a Python set.

Time complexity of Sets

Operators for Sets

Sets and frozen sets support the following operators:

Code Snippet to illustrate all Set operations in Python

# Python program to demonstrate working# of 
# Set in Python 

# Creating two sets 
set1 = set() 
set2 = set() 

# Adding elements to set1 
for i in range(1, 6): 
	set1.add(i) 

# Adding elements to set2 
for i in range(3, 8): 
	set2.add(i) 

print("Set1 = ", set1) 
print("Set2 = ", set2) 
print("\n") 

# Union of set1 and set2 
set3 = set1 | set2# set1.union(set2) 
print("Union of Set1 & Set2: Set3 = ", set3) 

# Intersection of set1 and set2 
set4 = set1 & set2# set1.intersection(set2) 
print("Intersection of Set1 & Set2: Set4 = ", set4) 
print("\n") 

# Checking relation between set3 and set4 
if set3 > set4: # set3.issuperset(set4) 
	print("Set3 is superset of Set4") 
elif set3 < set4: # set3.issubset(set4) 
	print("Set3 is subset of Set4") 
else : # set3 == set4 
	print("Set3 is same as Set4") 

# displaying relation between set4 and set3 
if set4 < set3: # set4.issubset(set3) 
	print("Set4 is subset of Set3") 
	print("\n") 

# difference between set3 and set4 
set5 = set3 - set4 
print("Elements in Set3 and not in Set4: Set5 = ", set5) 
print("\n") 

# checkv if set4 and set5 are disjoint sets 
if set4.isdisjoint(set5): 
	print("Set4 and Set5 have nothing in common\n") 

# Removing all the values of set5 
set5.clear() 

print("After applying clear on sets Set5: ") 
print("Set5 = ", set5)

Output:

('Set1 = ', set([1, 2, 3, 4, 5]))
('Set2 = ', set([3, 4, 5, 6, 7]))


('Union of Set1 & Set2: Set3 = ', set([1, 2, 3, 4, 5, 6, 7]))
('Intersection of Set1 & Set2: Set4 = ', set([3, 4, 5]))


Set3 is superset of Set4
Set4 is subset of Set3


('Elements in Set3 and not in Set4: Set5 = ', set([1, 2, 6, 7]))


Set4 and Set5 have nothing in common

After applying clear on sets Set5: 
('Set5 = ', set([]))

#python #web-development #machine-learning

What is GEEK

Buddha Community

Learn Python Programming - Sets in Python

Hermann Frami

1651383480

A Simple Wrapper Around Amplify AppSync Simulator

This serverless plugin is a wrapper for amplify-appsync-simulator made for testing AppSync APIs built with serverless-appsync-plugin.

Install

npm install serverless-appsync-simulator
# or
yarn add serverless-appsync-simulator

Usage

This plugin relies on your serverless yml file and on the serverless-offline plugin.

plugins:
  - serverless-dynamodb-local # only if you need dynamodb resolvers and you don't have an external dynamodb
  - serverless-appsync-simulator
  - serverless-offline

Note: Order is important serverless-appsync-simulator must go before serverless-offline

To start the simulator, run the following command:

sls offline start

You should see in the logs something like:

...
Serverless: AppSync endpoint: http://localhost:20002/graphql
Serverless: GraphiQl: http://localhost:20002
...

Configuration

Put options under custom.appsync-simulator in your serverless.yml file

| option | default | description | | ------------------------ | -------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------- | | apiKey | 0123456789 | When using API_KEY as authentication type, the key to authenticate to the endpoint. | | port | 20002 | AppSync operations port; if using multiple APIs, the value of this option will be used as a starting point, and each other API will have a port of lastPort + 10 (e.g. 20002, 20012, 20022, etc.) | | wsPort | 20003 | AppSync subscriptions port; if using multiple APIs, the value of this option will be used as a starting point, and each other API will have a port of lastPort + 10 (e.g. 20003, 20013, 20023, etc.) | | location | . (base directory) | Location of the lambda functions handlers. | | refMap | {} | A mapping of resource resolutions for the Ref function | | getAttMap | {} | A mapping of resource resolutions for the GetAtt function | | importValueMap | {} | A mapping of resource resolutions for the ImportValue function | | functions | {} | A mapping of external functions for providing invoke url for external fucntions | | dynamoDb.endpoint | http://localhost:8000 | Dynamodb endpoint. Specify it if you're not using serverless-dynamodb-local. Otherwise, port is taken from dynamodb-local conf | | dynamoDb.region | localhost | Dynamodb region. Specify it if you're connecting to a remote Dynamodb intance. | | dynamoDb.accessKeyId | DEFAULT_ACCESS_KEY | AWS Access Key ID to access DynamoDB | | dynamoDb.secretAccessKey | DEFAULT_SECRET | AWS Secret Key to access DynamoDB | | dynamoDb.sessionToken | DEFAULT_ACCESS_TOKEEN | AWS Session Token to access DynamoDB, only if you have temporary security credentials configured on AWS | | dynamoDb.* | | You can add every configuration accepted by DynamoDB SDK | | rds.dbName | | Name of the database | | rds.dbHost | | Database host | | rds.dbDialect | | Database dialect. Possible values (mysql | postgres) | | rds.dbUsername | | Database username | | rds.dbPassword | | Database password | | rds.dbPort | | Database port | | watch | - *.graphql
- *.vtl | Array of glob patterns to watch for hot-reloading. |

Example:

custom:
  appsync-simulator:
    location: '.webpack/service' # use webpack build directory
    dynamoDb:
      endpoint: 'http://my-custom-dynamo:8000'

Hot-reloading

By default, the simulator will hot-relad when changes to *.graphql or *.vtl files are detected. Changes to *.yml files are not supported (yet? - this is a Serverless Framework limitation). You will need to restart the simulator each time you change yml files.

Hot-reloading relies on watchman. Make sure it is installed on your system.

You can change the files being watched with the watch option, which is then passed to watchman as the match expression.

e.g.

custom:
  appsync-simulator:
    watch:
      - ["match", "handlers/**/*.vtl", "wholename"] # => array is interpreted as the literal match expression
      - "*.graphql"                                 # => string like this is equivalent to `["match", "*.graphql"]`

Or you can opt-out by leaving an empty array or set the option to false

Note: Functions should not require hot-reloading, unless you are using a transpiler or a bundler (such as webpack, babel or typescript), un which case you should delegate hot-reloading to that instead.

Resource CloudFormation functions resolution

This plugin supports some resources resolution from the Ref, Fn::GetAtt and Fn::ImportValue functions in your yaml file. It also supports some other Cfn functions such as Fn::Join, Fb::Sub, etc.

Note: Under the hood, this features relies on the cfn-resolver-lib package. For more info on supported cfn functions, refer to the documentation

Basic usage

You can reference resources in your functions' environment variables (that will be accessible from your lambda functions) or datasource definitions. The plugin will automatically resolve them for you.

provider:
  environment:
    BUCKET_NAME:
      Ref: MyBucket # resolves to `my-bucket-name`

resources:
  Resources:
    MyDbTable:
      Type: AWS::DynamoDB::Table
      Properties:
        TableName: myTable
      ...
    MyBucket:
      Type: AWS::S3::Bucket
      Properties:
        BucketName: my-bucket-name
    ...

# in your appsync config
dataSources:
  - type: AMAZON_DYNAMODB
    name: dynamosource
    config:
      tableName:
        Ref: MyDbTable # resolves to `myTable`

Override (or mock) values

Sometimes, some references cannot be resolved, as they come from an Output from Cloudformation; or you might want to use mocked values in your local environment.

In those cases, you can define (or override) those values using the refMap, getAttMap and importValueMap options.

refMap takes a mapping of resource name to value pairs
getAttMap takes a mapping of resource name to attribute/values pairs
importValueMap takes a mapping of import name to values pairs

Example:

custom:
  appsync-simulator:
    refMap:
      # Override `MyDbTable` resolution from the previous example.
      MyDbTable: 'mock-myTable'
    getAttMap:
      # define ElasticSearchInstance DomainName
      ElasticSearchInstance:
        DomainEndpoint: 'localhost:9200'
    importValueMap:
      other-service-api-url: 'https://other.api.url.com/graphql'

# in your appsync config
dataSources:
  - type: AMAZON_ELASTICSEARCH
    name: elasticsource
    config:
      # endpoint resolves as 'http://localhost:9200'
      endpoint:
        Fn::Join:
          - ''
          - - https://
            - Fn::GetAtt:
                - ElasticSearchInstance
                - DomainEndpoint

Key-value mock notation

In some special cases you will need to use key-value mock nottation. Good example can be case when you need to include serverless stage value (${self:provider.stage}) in the import name.

This notation can be used with all mocks - refMap, getAttMap and importValueMap

provider:
  environment:
    FINISH_ACTIVITY_FUNCTION_ARN:
      Fn::ImportValue: other-service-api-${self:provider.stage}-url

custom:
  serverless-appsync-simulator:
    importValueMap:
      - key: other-service-api-${self:provider.stage}-url
        value: 'https://other.api.url.com/graphql'

Limitations

This plugin only tries to resolve the following parts of the yml tree:

provider.environment
functions[*].environment
custom.appSync

If you have the need of resolving others, feel free to open an issue and explain your use case.

For now, the supported resources to be automatically resovled by Ref: are:

DynamoDb tables
S3 Buckets

Feel free to open a PR or an issue to extend them as well.

External functions

When a function is not defined withing the current serverless file you can still call it by providing an invoke url which should point to a REST method. Make sure you specify "get" or "post" for the method. Default is "get", but you probably want "post".

custom:
  appsync-simulator:
    functions:
      addUser:
        url: http://localhost:3016/2015-03-31/functions/addUser/invocations
        method: post
      addPost:
        url: https://jsonplaceholder.typicode.com/posts
        method: post

Supported Resolver types

This plugin supports resolvers implemented by amplify-appsync-simulator, as well as custom resolvers.

From Aws Amplify:

NONE
AWS_LAMBDA
AMAZON_DYNAMODB
PIPELINE

Implemented by this plugin

AMAZON_ELASTIC_SEARCH
HTTP
RELATIONAL_DATABASE

Relational Database

Sample VTL for a create mutation

#set( $cols = [] )
#set( $vals = [] )
#foreach( $entry in $ctx.args.input.keySet() )
  #set( $regex = "([a-z])([A-Z]+)")
  #set( $replacement = "$1_$2")
  #set( $toSnake = $entry.replaceAll($regex, $replacement).toLowerCase() )
  #set( $discard = $cols.add("$toSnake") )
  #if( $util.isBoolean($ctx.args.input[$entry]) )
      #if( $ctx.args.input[$entry] )
        #set( $discard = $vals.add("1") )
      #else
        #set( $discard = $vals.add("0") )
      #end
  #else
      #set( $discard = $vals.add("'$ctx.args.input[$entry]'") )
  #end
#end
#set( $valStr = $vals.toString().replace("[","(").replace("]",")") )
#set( $colStr = $cols.toString().replace("[","(").replace("]",")") )
#if ( $valStr.substring(0, 1) != '(' )
  #set( $valStr = "($valStr)" )
#end
#if ( $colStr.substring(0, 1) != '(' )
  #set( $colStr = "($colStr)" )
#end
{
  "version": "2018-05-29",
  "statements":   ["INSERT INTO <name-of-table> $colStr VALUES $valStr", "SELECT * FROM    <name-of-table> ORDER BY id DESC LIMIT 1"]
}

Sample VTL for an update mutation

#set( $update = "" )
#set( $equals = "=" )
#foreach( $entry in $ctx.args.input.keySet() )
  #set( $cur = $ctx.args.input[$entry] )
  #set( $regex = "([a-z])([A-Z]+)")
  #set( $replacement = "$1_$2")
  #set( $toSnake = $entry.replaceAll($regex, $replacement).toLowerCase() )
  #if( $util.isBoolean($cur) )
      #if( $cur )
        #set ( $cur = "1" )
      #else
        #set ( $cur = "0" )
      #end
  #end
  #if ( $util.isNullOrEmpty($update) )
      #set($update = "$toSnake$equals'$cur'" )
  #else
      #set($update = "$update,$toSnake$equals'$cur'" )
  #end
#end
{
  "version": "2018-05-29",
  "statements":   ["UPDATE <name-of-table> SET $update WHERE id=$ctx.args.input.id", "SELECT * FROM <name-of-table> WHERE id=$ctx.args.input.id"]
}

Sample resolver for delete mutation

{
  "version": "2018-05-29",
  "statements":   ["UPDATE <name-of-table> set deleted_at=NOW() WHERE id=$ctx.args.id", "SELECT * FROM <name-of-table> WHERE id=$ctx.args.id"]
}

Sample mutation response VTL with support for handling AWSDateTime

#set ( $index = -1)
#set ( $result = $util.parseJson($ctx.result) )
#set ( $meta = $result.sqlStatementResults[1].columnMetadata)
#foreach ($column in $meta)
    #set ($index = $index + 1)
    #if ( $column["typeName"] == "timestamptz" )
        #set ($time = $result["sqlStatementResults"][1]["records"][0][$index]["stringValue"] )
        #set ( $nowEpochMillis = $util.time.parseFormattedToEpochMilliSeconds("$time.substring(0,19)+0000", "yyyy-MM-dd HH:mm:ssZ") )
        #set ( $isoDateTime = $util.time.epochMilliSecondsToISO8601($nowEpochMillis) )
        $util.qr( $result["sqlStatementResults"][1]["records"][0][$index].put("stringValue", "$isoDateTime") )
    #end
#end
#set ( $res = $util.parseJson($util.rds.toJsonString($util.toJson($result)))[1][0] )
#set ( $response = {} )
#foreach($mapKey in $res.keySet())
    #set ( $s = $mapKey.split("_") )
    #set ( $camelCase="" )
    #set ( $isFirst=true )
    #foreach($entry in $s)
        #if ( $isFirst )
          #set ( $first = $entry.substring(0,1) )
        #else
          #set ( $first = $entry.substring(0,1).toUpperCase() )
        #end
        #set ( $isFirst=false )
        #set ( $stringLength = $entry.length() )
        #set ( $remaining = $entry.substring(1, $stringLength) )
        #set ( $camelCase = "$camelCase$first$remaining" )
    #end
    $util.qr( $response.put("$camelCase", $res[$mapKey]) )
#end
$utils.toJson($response)

Using Variable Map

Variable map support is limited and does not differentiate numbers and strings data types, please inject them directly if needed.

Will be escaped properly: null, true, and false values.

{
  "version": "2018-05-29",
  "statements":   [
    "UPDATE <name-of-table> set deleted_at=NOW() WHERE id=:ID",
    "SELECT * FROM <name-of-table> WHERE id=:ID and unix_timestamp > $ctx.args.newerThan"
  ],
  variableMap: {
    ":ID": $ctx.args.id,
##    ":TIMESTAMP": $ctx.args.newerThan -- This will be handled as a string!!!
  }
}

Requires

serverless framework
serverless-appsync-plugin
serverless-offline
serverless-dynamodb-local (when using dynamodb resolvers only)
watchman (if Hot-reloading is desactivated it is not required)

Author: Serverless-appsync
Source Code: https://github.com/serverless-appsync/serverless-appsync-simulator
License: MIT License

#serverless #sync #graphql

Ray Patel

1625843760

Python Packages in SQL Server – Get Started with SQL Server Machine Learning Services

Introduction

When installing Machine Learning Services in SQL Server by default few Python Packages are installed. In this article, we will have a look on how to get those installed python package information.

Python Packages

When we choose Python as Machine Learning Service during installation, the following packages are installed in SQL Server,

revoscalepy – This Microsoft Python package is used for remote compute contexts, streaming, parallel execution of rx functions for data import and transformation, modeling, visualization, and analysis.
microsoftml – This is another Microsoft Python package which adds machine learning algorithms in Python.
Anaconda 4.2 – Anaconda is an opensource Python package

#machine learning #sql server #executing python in sql server #machine learning using python #machine learning with sql server #ml in sql server using python #python in sql server ml #python packages #python packages for machine learning services #sql server machine learning services

Zakary Goyette

1602154800

Learning Python: The Prompt, Then Read Template

In my last article I discussed how to use the Input-Process-Output template as a general guide to how programs should be structured in Python. To review, most Python programs will consist of three steps — getting input into the program, processing the input in some way, and outputting the results of the processing.

In this article, I’m going to focus on one part of that step — getting input into a program — by prompting the user to enter some data and then reading the data into the program. This is a mostly straightforward process except for some data conversions that have to occur when you are inputting numbers.

Defining a Prompt

Let’s start by defining the word prompt. A prompt is a message to the user of your program telling them what they are supposed to be entering into the program. A prompt needs to be descriptive but doesn’t have to be overly detailed.

For example, if your program needs the user to enter their name, you can use a prompt like this:

Enter your name:

However, you may need to be more specific if you want the user to enter their first name and their last name separately. The more appropriate prompt in this case might really be two prompts:

Enter your first name:
Enter your last name:

#learn-to-code #learn-python-programming #python #learning-python #python-programming

Biju Augustian

1574340826

Learn Python Programming

Description
Learn Python Programming

Learn Python Programming and increase your python programming skills with Coder Kovid.

Python is the highest growing programming language in this era. You can use Python to do everything like, web development, software development, cognitive development, machine learning, artificial intelligence, etc. You should learn python programming and increase your skills of programming.

In this course of learn python programming you don’t need any prior programming knowledge. Every beginner can start with.

Basic knowledge
No prior knowledge needed to learn this course
What will you learn
Write Basic Syntax of Python Programming
Create Basic Real World Application
Program in a fluent manner
Get Familiar in Programming Environment

#Learn Python Programming # Learn Python #Python Programming # Programming

Biju Augustian

1574339728

Learn Programming With Python In 100 Steps

Description
We love Programming. Our aim with this course is to create a love for Programming.

Python is one of the most popular programming languages. Python offers both object oriented and structural programming features.

We take an hands-on approach using a combination of Python Shell and PyCharm as an IDE to illustrate more than 150 Python Coding Exercises, Puzzles and Code Examples.

In more than 150 Steps, we explore the most important Python Programming Language Features

Basics of Python Programming - Expressions, Variables and Printing Output
Python Operators - Python Assignment Operator, Relational and Logical Operators, Short Circuit Operators
Python Conditionals and If Statement
Methods - Parameters, Arguments and Return Values
An Overview Of Python Platform
Object Oriented Programming - Class, Object, State and Behavior
Basics of OOPS - Encapsulation, Inheritance and Abstract Class.
Basics about Python Data Types
Basics about Python Built in Modules
Conditionals with Python - If Else Statement, Nested If Else
Loops - For Loop, While Loop in Python, Break and Continue
Immutablity of Python Basic Types
Python Data Structures - List, Set, Dictionary and Tuples
Introduction to Variable Arguments
Basics of Designing a Class - Class, Object, State and Behavior. Deciding State and Constructors.
Introduction to Exception Handling - Your Thought Process during Exception Handling. try, except, else and finally. Exception Hierarchy. Throwing an Exception. Creating and Throwing a Custom Exception

To read more:

#Programming with Python #Learn Programming #Learn Programming with Python #Python