1613619101
What is Polymath Network (POLY) | What is Polymath Network token | What is POLY token
About Polymath
Polymath (POLY) is creating a global platform for issuing and investing in securities tokens. Polymath’s standard for blockchain security tokens aims to integrate the necessary regulatory requirements into smart contracts and comply with regulations. The project simplifies the legal process of creating and selling security tokens. It establishes a new token standard (ST20) and enforces compliance by whitelisting authorized investors and their Ethereum wallet addresses. The POLY token is used for payments on the platform, which facilitates exchanges between issuers, investors, service providers, and developers.
Polymath simplifies the legal process of creating and selling security tokens. It makes a new token standard, the ST20, and enforces government compliance. Only a “list of authorized investors and their Ethereum wallet addresses” can hold ST20 tokens. Therefore, token issuers don’t need to worry about the legal implications of their security falling into the wrong hands.
In order to launch a legally compliant token, the Polymath platform brings together issuers, legal delegates, smart contract developers, KYC verification, and a decentralized exchange. All transactions on the Polymath platform take place using the native POLY token. Due to the continuing government crackdown on initial coin offerings, Polymath hopes to instead provide legal Security Token Offerings.
Polymesh, a blockchain built for security tokens, will be going live in March 2021. Get a behind-the-scenes look at the brand-new Token Studio in this quick 3-minute demo.
Driving efficient capital markets
We relentlessly focus on thetechnological backbone for security tokens. Whether configuring tokens using the Polymath Token Studio or integrating our broker-dealer toolkit into your offering, Polymath’s technology brings efficiency and transparency to security token management.
Open source.
Transparency is at the core of all Polymath products.
Standardized.
ERC1400 standard ensures the token’s code meets specific requirements in order to minimize technical due diligence.
Modular.
Build tokens your way, for your jurisdictions. Polymath modules let you customize your offering, manage your token, and provide corporate actions.
Automated.
Polymath lets issuers set rules around who can hold their token, how it can be transferred and other jurisdictional requirements. From there, the system will automate rule enforcement.
End-to-end.
Polymath’s self-serve token creation and management technology integrates smoothly with a large ecosystem of custodians, broker-dealers, legal firms, cap table management providers, token sale platforms, KYC/AML providers and others so that issuers can create a bespoke end-to-end solution.
Purpose-built.
Polymesh is an institutional-grade blockchain built specifically for security tokens that solves the inherent challenges with public infrastructure around identity, compliance, confidentiality, and governance.
How Does Polymath Work?
Token Supply and Sustainability
Overall, there are one billion POLY tokens that will ever exist. You could have signed up for the Polymath airdrop before January 10th, 2018. Subsequently, airdrop participants received 240 million tokens.
At this point, the Polymath team retains the rest for future use. If you missed the airdrop, then you’ll need to trade for POLY on an exchange.
Polymath Securities Token Platform
The token platform involves three layers: an application layer, a legal layer, and the protocol layer to ensure your token is compliant and stays compliant. These layers aim to reduce the legal complexity and ambiguity surrounding securities while also minimizing fees and improving the liquidity of assets.
Launching a Security Token
Polymath asks for the following information to instantly create your ST20 standard security token.
- Legal Name:
- Legal Entity Type:
- Type of Security:
- Project Description:
- Logo:
- TOKEN CHARACTERISTICS
- Voting Rights:
- Dividend:
- Dividend Frequency:
- Corporate Governance:
- Governance Integration Partner:
- Additional Features:
- TOKEN ALLOCATION
- Tokens to Create:
- Percentage of Tokens Held by Company:
- Percentage of Company Equity Distributed With Tokens:
- Price per Token in USD:
- LEGAL DETAILS
- Issuing Jurisdiction:
- Offering Security To:
- Investors Must be Accredited:
- Investor KYC Needed:
- KYC Integration Partner:
- Tokens Freely Tradable:
- CONTACT DETAILS
- Contact Name:
- Position at Company:
- Contact Phone Number:
- Contact Email:
- Permit Contact from Polymath:
Choosing a Legal Delegate
After creating your new token, it’s still non-transferrable until a legal delegate confirms “that the steps have been completed for the token to be issued.”
You’ll receive several bids from legal delegates, lawyers, but it’s up to you to perform due diligence and compare their fees. Once you choose a legal delegate, you send POLY for their fee to a smart contract and begin working together on the Polymath platform by securely sending necessary documents and working through the compliance process.
Part of the process entails delegates working with developers to build a smart contract specifically for your token. Then, the smart contract enforces investor requirements such as jurisdiction of investors, type of offering, hold time before tokens can be resold etc. After the necessary documents are sent and the smart contract completed, the legal delegate will set the address of your initial offering contract. You’re finally ready to start trading.
Becoming an Investor
If you’d like to purchase a token on Polymath’s platform, you first need to have your identity and accreditation status confirmed by a KYC (know your customer) provider. You can search and choose a KYC provider on the Polymath network. Next, you’d send the required number of POLY tokens to escrow until your KYC process is completed. After submitting the necessary documents, the KYC provider can send information to a Polymath smart contract specifying details on your ability to buy securities. The information stored in the smart contract will also determine the amount of money you can invest in securities and where you can trade them.
Security Tokens
While most tokens are easily tradable on exchanges, security tokens follow different rules due to numerous legal implications. It’s important to know who owns a security token at all times due to securities potentially providing voting rights, dividends, or other income with tax implications. Exchanges currently avoid listing any token potentially considered a security to avoid security regulators.
However, with a token created through Polymath, the smart contracts verify who can buy and sell the token. Only investors authorized under Polymath’s KYC providers will be able to hold the token. This could, in theory, eliminate the fees associated with centralized exchanges that ensure security compliance.
The Polymath investment process
Learn more about Polymesh Testnet II
Today marks the release of Polymesh Testnet II: Alcyone. Much like it’s predecessor Aldebaran, Alcyone is the name of a star within the Taurus constellation, a nice nod to our Polymath brand bull.
Whereas Testnet I was predominantly user-facing, much of what Alcyone brings to life is under the hood of Polymesh, with new enterprise features to help institutional users clear the roadblocks found on public, permissionless blockchains.
Here’s what you can expect from Alcyone:
Confidential Assets Infrastructure
On public chains, anyone can see the contents of a public address. While users can easily make transactions and balances confidential with layer-2 solutions, they come with an unworkable compromise — to add confidentiality, they must sacrifice compliance. Polymesh enables users to issue confidential assets while still adhering to compliance criteria in an automated way. The Alcyone release is simply the first step to implementing confidentiality with much more on the way — look forward to a technical white paper on the specifics of how Polymesh manages confidentiality later in the fall.
Portfolios
Portfolios help users group assets in a way that makes sense for them, letting them organize large numbers of assets in a functional and reportable way. Alcyone testers can start using portfolios right away and can look forward to dashboarding capabilities in Q4.
Permissions
With the Alcyone testnet, users will be able to define granular permissions for who can perform which actions on the blockchain. When combined with the portfolios function, users are able to segment responsibilities and efficiently control access to assets.
Attestation Management SDK Documentation
Attestations are the feature that ties user identity and compliance together, allowing the issuer to automate the compliance framework for their security token. More specifically, they are the on-chain confirmation that an attribute of a user’s identity is true. Sometimes attestations are based on the token (e.g. buy or sell lockups tied to when a token was issued). Other times, they’re based on the unique identity of the user and need to be verified through third-party service providers (e.g. residency; accreditation). SDK documentation makes it simple for enterprise users and attestation providers to implement this feature.
Importantly, the move from testnet I to II marks a reset of the chain — unless absolutely necessary, it’s the only one we plan to do. For us, testnet is as much about refining our internal processes as it is about ironing out bugs in the code. We know that it’s important to quickly get new features into the hands of users, but we need to be able to do this through smooth network upgrades. From here on out, we plan to conduct regular upgrades to the chain rather than doing another full reset (of course, we’ll be launching a brand new chain for mainnet). This helps us ensure that we can deliver a seamless upgrade process once we get to mainnet, and it gives users more immediate access to new features as they’re developed.
Look out for regular new feature releases as we head towards mainnet in Q1 2021.
Looking for more information…
☞ Website ☞ Explorer ☞ Explorer 2 ☞ Whitepaper ☞ Source Code ☞ Social Channel ☞ Social Channel 2 ☞ Message Board ☞ Message Board 2 ☞ Coinmarketcap
Would you like to earn POLY right now! ☞ CLICK HERE
Top exchanges for token-coin trading. Follow instructions and make unlimited money
☞ Binance ☞ Bittrex ☞ Poloniex ☞ Bitfinex ☞ Huobi
Thank for visiting and reading this article! I’m highly appreciate your actions! Please share if you liked it!
#blockchain #bitcoin #polymath network #poly
What is GEEK
Buddha Community
1613619101
What is Polymath Network (POLY) | What is Polymath Network token | What is POLY token
About Polymath
Polymath (POLY) is creating a global platform for issuing and investing in securities tokens. Polymath’s standard for blockchain security tokens aims to integrate the necessary regulatory requirements into smart contracts and comply with regulations. The project simplifies the legal process of creating and selling security tokens. It establishes a new token standard (ST20) and enforces compliance by whitelisting authorized investors and their Ethereum wallet addresses. The POLY token is used for payments on the platform, which facilitates exchanges between issuers, investors, service providers, and developers.
Polymath simplifies the legal process of creating and selling security tokens. It makes a new token standard, the ST20, and enforces government compliance. Only a “list of authorized investors and their Ethereum wallet addresses” can hold ST20 tokens. Therefore, token issuers don’t need to worry about the legal implications of their security falling into the wrong hands.
In order to launch a legally compliant token, the Polymath platform brings together issuers, legal delegates, smart contract developers, KYC verification, and a decentralized exchange. All transactions on the Polymath platform take place using the native POLY token. Due to the continuing government crackdown on initial coin offerings, Polymath hopes to instead provide legal Security Token Offerings.
Polymesh, a blockchain built for security tokens, will be going live in March 2021. Get a behind-the-scenes look at the brand-new Token Studio in this quick 3-minute demo.
Driving efficient capital markets
We relentlessly focus on thetechnological backbone for security tokens. Whether configuring tokens using the Polymath Token Studio or integrating our broker-dealer toolkit into your offering, Polymath’s technology brings efficiency and transparency to security token management.
Open source.
Transparency is at the core of all Polymath products.
Standardized.
ERC1400 standard ensures the token’s code meets specific requirements in order to minimize technical due diligence.
Modular.
Build tokens your way, for your jurisdictions. Polymath modules let you customize your offering, manage your token, and provide corporate actions.
Automated.
Polymath lets issuers set rules around who can hold their token, how it can be transferred and other jurisdictional requirements. From there, the system will automate rule enforcement.
End-to-end.
Polymath’s self-serve token creation and management technology integrates smoothly with a large ecosystem of custodians, broker-dealers, legal firms, cap table management providers, token sale platforms, KYC/AML providers and others so that issuers can create a bespoke end-to-end solution.
Purpose-built.
Polymesh is an institutional-grade blockchain built specifically for security tokens that solves the inherent challenges with public infrastructure around identity, compliance, confidentiality, and governance.
How Does Polymath Work?
Token Supply and Sustainability
Overall, there are one billion POLY tokens that will ever exist. You could have signed up for the Polymath airdrop before January 10th, 2018. Subsequently, airdrop participants received 240 million tokens.
At this point, the Polymath team retains the rest for future use. If you missed the airdrop, then you’ll need to trade for POLY on an exchange.
Polymath Securities Token Platform
The token platform involves three layers: an application layer, a legal layer, and the protocol layer to ensure your token is compliant and stays compliant. These layers aim to reduce the legal complexity and ambiguity surrounding securities while also minimizing fees and improving the liquidity of assets.
Launching a Security Token
Polymath asks for the following information to instantly create your ST20 standard security token.
- Legal Name:
- Legal Entity Type:
- Type of Security:
- Project Description:
- Logo:
- TOKEN CHARACTERISTICS
- Voting Rights:
- Dividend:
- Dividend Frequency:
- Corporate Governance:
- Governance Integration Partner:
- Additional Features:
- TOKEN ALLOCATION
- Tokens to Create:
- Percentage of Tokens Held by Company:
- Percentage of Company Equity Distributed With Tokens:
- Price per Token in USD:
- LEGAL DETAILS
- Issuing Jurisdiction:
- Offering Security To:
- Investors Must be Accredited:
- Investor KYC Needed:
- KYC Integration Partner:
- Tokens Freely Tradable:
- CONTACT DETAILS
- Contact Name:
- Position at Company:
- Contact Phone Number:
- Contact Email:
- Permit Contact from Polymath:
Choosing a Legal Delegate
After creating your new token, it’s still non-transferrable until a legal delegate confirms “that the steps have been completed for the token to be issued.”
You’ll receive several bids from legal delegates, lawyers, but it’s up to you to perform due diligence and compare their fees. Once you choose a legal delegate, you send POLY for their fee to a smart contract and begin working together on the Polymath platform by securely sending necessary documents and working through the compliance process.
Part of the process entails delegates working with developers to build a smart contract specifically for your token. Then, the smart contract enforces investor requirements such as jurisdiction of investors, type of offering, hold time before tokens can be resold etc. After the necessary documents are sent and the smart contract completed, the legal delegate will set the address of your initial offering contract. You’re finally ready to start trading.
Becoming an Investor
If you’d like to purchase a token on Polymath’s platform, you first need to have your identity and accreditation status confirmed by a KYC (know your customer) provider. You can search and choose a KYC provider on the Polymath network. Next, you’d send the required number of POLY tokens to escrow until your KYC process is completed. After submitting the necessary documents, the KYC provider can send information to a Polymath smart contract specifying details on your ability to buy securities. The information stored in the smart contract will also determine the amount of money you can invest in securities and where you can trade them.
Security Tokens
While most tokens are easily tradable on exchanges, security tokens follow different rules due to numerous legal implications. It’s important to know who owns a security token at all times due to securities potentially providing voting rights, dividends, or other income with tax implications. Exchanges currently avoid listing any token potentially considered a security to avoid security regulators.
However, with a token created through Polymath, the smart contracts verify who can buy and sell the token. Only investors authorized under Polymath’s KYC providers will be able to hold the token. This could, in theory, eliminate the fees associated with centralized exchanges that ensure security compliance.
The Polymath investment process
Learn more about Polymesh Testnet II
Today marks the release of Polymesh Testnet II: Alcyone. Much like it’s predecessor Aldebaran, Alcyone is the name of a star within the Taurus constellation, a nice nod to our Polymath brand bull.
Whereas Testnet I was predominantly user-facing, much of what Alcyone brings to life is under the hood of Polymesh, with new enterprise features to help institutional users clear the roadblocks found on public, permissionless blockchains.
Here’s what you can expect from Alcyone:
Confidential Assets Infrastructure
On public chains, anyone can see the contents of a public address. While users can easily make transactions and balances confidential with layer-2 solutions, they come with an unworkable compromise — to add confidentiality, they must sacrifice compliance. Polymesh enables users to issue confidential assets while still adhering to compliance criteria in an automated way. The Alcyone release is simply the first step to implementing confidentiality with much more on the way — look forward to a technical white paper on the specifics of how Polymesh manages confidentiality later in the fall.
Portfolios
Portfolios help users group assets in a way that makes sense for them, letting them organize large numbers of assets in a functional and reportable way. Alcyone testers can start using portfolios right away and can look forward to dashboarding capabilities in Q4.
Permissions
With the Alcyone testnet, users will be able to define granular permissions for who can perform which actions on the blockchain. When combined with the portfolios function, users are able to segment responsibilities and efficiently control access to assets.
Attestation Management SDK Documentation
Attestations are the feature that ties user identity and compliance together, allowing the issuer to automate the compliance framework for their security token. More specifically, they are the on-chain confirmation that an attribute of a user’s identity is true. Sometimes attestations are based on the token (e.g. buy or sell lockups tied to when a token was issued). Other times, they’re based on the unique identity of the user and need to be verified through third-party service providers (e.g. residency; accreditation). SDK documentation makes it simple for enterprise users and attestation providers to implement this feature.
Importantly, the move from testnet I to II marks a reset of the chain — unless absolutely necessary, it’s the only one we plan to do. For us, testnet is as much about refining our internal processes as it is about ironing out bugs in the code. We know that it’s important to quickly get new features into the hands of users, but we need to be able to do this through smooth network upgrades. From here on out, we plan to conduct regular upgrades to the chain rather than doing another full reset (of course, we’ll be launching a brand new chain for mainnet). This helps us ensure that we can deliver a seamless upgrade process once we get to mainnet, and it gives users more immediate access to new features as they’re developed.
Look out for regular new feature releases as we head towards mainnet in Q1 2021.
Looking for more information…
☞ Website ☞ Explorer ☞ Explorer 2 ☞ Whitepaper ☞ Source Code ☞ Social Channel ☞ Social Channel 2 ☞ Message Board ☞ Message Board 2 ☞ Coinmarketcap
Would you like to earn POLY right now! ☞ CLICK HERE
Top exchanges for token-coin trading. Follow instructions and make unlimited money
☞ Binance ☞ Bittrex ☞ Poloniex ☞ Bitfinex ☞ Huobi
Thank for visiting and reading this article! I’m highly appreciate your actions! Please share if you liked it!
#blockchain #bitcoin #polymath network #poly
1659601560
Words Counted: A Ruby Natural Language Processor.
WordsCounted
We are all in the gutter, but some of us are looking at the stars.
-- Oscar Wilde
WordsCounted is a Ruby NLP (natural language processor). WordsCounted lets you implement powerful tokensation strategies with a very flexible tokeniser class.
Are you using WordsCounted to do something interesting? Please tell me about it.
Demo
Visit this website for one example of what you can do with WordsCounted.
Features
- Out of the box, get the following data from any string or readable file, or URL:
- Token count and unique token count
- Token densities, frequencies, and lengths
- Char count and average chars per token
- The longest tokens and their lengths
- The most frequent tokens and their frequencies.
- A flexible way to exclude tokens from the tokeniser. You can pass a string, regexp, symbol, lambda, or an array of any combination of those types for powerful tokenisation strategies.
- Pass your own regexp rules to the tokeniser if you prefer. The default regexp filters special characters but keeps hyphens and apostrophes. It also plays nicely with diacritics (UTF and unicode characters): Bayrūt is treated as
["Bayrūt"]and not["Bayr", "ū", "t"], for example. - Opens and reads files. Pass in a file path or a url instead of a string.
Installation
Add this line to your application's Gemfile:
gem 'words_counted'
And then execute:
$ bundle
Or install it yourself as:
$ gem install words_counted
Usage
Pass in a string or a file path, and an optional filter and/or regexp.
counter = WordsCounted.count(
"We are all in the gutter, but some of us are looking at the stars."
)
# Using a file
counter = WordsCounted.from_file("path/or/url/to/my/file.txt")
.count and .from_file are convenience methods that take an input, tokenise it, and return an instance of WordsCounted::Counter initialized with the tokens. The WordsCounted::Tokeniser and WordsCounted::Counter classes can be used alone, however.
API
WordsCounted
WordsCounted.count(input, options = {})
Tokenises input and initializes a WordsCounted::Counter object with the resulting tokens.
counter = WordsCounted.count("Hello Beirut!")
Accepts two options: exclude and regexp. See Excluding tokens from the analyser and Passing in a custom regexp respectively.
WordsCounted.from_file(path, options = {})
Reads and tokenises a file, and initializes a WordsCounted::Counter object with the resulting tokens.
counter = WordsCounted.from_file("hello_beirut.txt")
Accepts the same options as .count.
Tokeniser
The tokeniser allows you to tokenise text in a variety of ways. You can pass in your own rules for tokenisation, and apply a powerful filter with any combination of rules as long as they can boil down into a lambda.
Out of the box the tokeniser includes only alpha chars. Hyphenated tokens and tokens with apostrophes are considered a single token.
#tokenise([pattern: TOKEN_REGEXP, exclude: nil])
tokeniser = WordsCounted::Tokeniser.new("Hello Beirut!").tokenise
# With `exclude`
tokeniser = WordsCounted::Tokeniser.new("Hello Beirut!").tokenise(exclude: "hello")
# With `pattern`
tokeniser = WordsCounted::Tokeniser.new("I <3 Beirut!").tokenise(pattern: /[a-z]/i)
See Excluding tokens from the analyser and Passing in a custom regexp for more information.
Counter
The WordsCounted::Counter class allows you to collect various statistics from an array of tokens.
#token_count
Returns the token count of a given string.
counter.token_count #=> 15
#token_frequency
Returns a sorted (unstable) two-dimensional array where each element is a token and its frequency. The array is sorted by frequency in descending order.
counter.token_frequency
[
["the", 2],
["are", 2],
["we", 1],
# ...
["all", 1]
]
#most_frequent_tokens
Returns a hash where each key-value pair is a token and its frequency.
counter.most_frequent_tokens
{ "are" => 2, "the" => 2 }
#token_lengths
Returns a sorted (unstable) two-dimentional array where each element contains a token and its length. The array is sorted by length in descending order.
counter.token_lengths
[
["looking", 7],
["gutter", 6],
["stars", 5],
# ...
["in", 2]
]
#longest_tokens
Returns a hash where each key-value pair is a token and its length.
counter.longest_tokens
{ "looking" => 7 }
#token_density([ precision: 2 ])
Returns a sorted (unstable) two-dimentional array where each element contains a token and its density as a float, rounded to a precision of two. The array is sorted by density in descending order. It accepts a precision argument, which must be a float.
counter.token_density
[
["are", 0.13],
["the", 0.13],
["but", 0.07 ],
# ...
["we", 0.07 ]
]
#char_count
Returns the char count of tokens.
counter.char_count #=> 76
#average_chars_per_token([ precision: 2 ])
Returns the average char count per token rounded to two decimal places. Accepts a precision argument which defaults to two. Precision must be a float.
counter.average_chars_per_token #=> 4
#uniq_token_count
Returns the number of unique tokens.
counter.uniq_token_count #=> 13
Excluding tokens from the tokeniser
You can exclude anything you want from the input by passing the exclude option. The exclude option accepts a variety of filters and is extremely flexible.
- A space-delimited string. The filter will normalise the string.
- A regular expression.
- A lambda.
- A symbol that names a predicate method. For example
:odd?. - An array of any combination of the above.
tokeniser =
WordsCounted::Tokeniser.new(
"Magnificent! That was magnificent, Trevor."
)
# Using a string
tokeniser.tokenise(exclude: "was magnificent")
# => ["that", "trevor"]
# Using a regular expression
tokeniser.tokenise(exclude: /trevor/)
# => ["magnificent", "that", "was", "magnificent"]
# Using a lambda
tokeniser.tokenise(exclude: ->(t) { t.length < 4 })
# => ["magnificent", "that", "magnificent", "trevor"]
# Using symbol
tokeniser = WordsCounted::Tokeniser.new("Hello! محمد")
tokeniser.tokenise(exclude: :ascii_only?)
# => ["محمد"]
# Using an array
tokeniser = WordsCounted::Tokeniser.new(
"Hello! اسماءنا هي محمد، كارولينا، سامي، وداني"
)
tokeniser.tokenise(
exclude: [:ascii_only?, /محمد/, ->(t) { t.length > 6}, "و"]
)
# => ["هي", "سامي", "وداني"]
Passing in a custom regexp
The default regexp accounts for letters, hyphenated tokens, and apostrophes. This means twenty-one is treated as one token. So is Mohamad's.
/[\p{Alpha}\-']+/
You can pass your own criteria as a Ruby regular expression to split your string as desired.
For example, if you wanted to include numbers, you can override the regular expression:
counter = WordsCounted.count("Numbers 1, 2, and 3", pattern: /[\p{Alnum}\-']+/)
counter.tokens
#=> ["numbers", "1", "2", "and", "3"]
Opening and reading files
Use the from_file method to open files. from_file accepts the same options as .count. The file path can be a URL.
counter = WordsCounted.from_file("url/or/path/to/file.text")
Gotchas
A hyphen used in leu of an em or en dash will form part of the token. This affects the tokeniser algorithm.
counter = WordsCounted.count("How do you do?-you are well, I see.")
counter.token_frequency
[
["do", 2],
["how", 1],
["you", 1],
["-you", 1], # WTF, mate!
["are", 1],
# ...
]
In this example -you and you are separate tokens. Also, the tokeniser does not include numbers by default. Remember that you can pass your own regular expression if the default behaviour does not fit your needs.
A note on case sensitivity
The program will normalise (downcase) all incoming strings for consistency and filters.
Roadmap
Ability to open URLs
def self.from_url
# open url and send string here after removing html
end
Contributors
See contributors.
Contributing
- Fork it
- Create your feature branch (
git checkout -b my-new-feature) - Commit your changes (
git commit -am 'Add some feature') - Push to the branch (
git push origin my-new-feature) - Create new Pull Request
Author: abitdodgy
Source code: https://github.com/abitdodgy/words_counted
License: MIT license
#ruby #ruby-on-rails
1658068560
WordsCounted: A Ruby Natural Language Processor
WordsCounted
We are all in the gutter, but some of us are looking at the stars.
-- Oscar Wilde
WordsCounted is a Ruby NLP (natural language processor). WordsCounted lets you implement powerful tokensation strategies with a very flexible tokeniser class.
Features
- Out of the box, get the following data from any string or readable file, or URL:
- Token count and unique token count
- Token densities, frequencies, and lengths
- Char count and average chars per token
- The longest tokens and their lengths
- The most frequent tokens and their frequencies.
- A flexible way to exclude tokens from the tokeniser. You can pass a string, regexp, symbol, lambda, or an array of any combination of those types for powerful tokenisation strategies.
- Pass your own regexp rules to the tokeniser if you prefer. The default regexp filters special characters but keeps hyphens and apostrophes. It also plays nicely with diacritics (UTF and unicode characters): Bayrūt is treated as
["Bayrūt"]and not["Bayr", "ū", "t"], for example. - Opens and reads files. Pass in a file path or a url instead of a string.
Installation
Add this line to your application's Gemfile:
gem 'words_counted'
And then execute:
$ bundle
Or install it yourself as:
$ gem install words_counted
Usage
Pass in a string or a file path, and an optional filter and/or regexp.
counter = WordsCounted.count(
"We are all in the gutter, but some of us are looking at the stars."
)
# Using a file
counter = WordsCounted.from_file("path/or/url/to/my/file.txt")
.count and .from_file are convenience methods that take an input, tokenise it, and return an instance of WordsCounted::Counter initialized with the tokens. The WordsCounted::Tokeniser and WordsCounted::Counter classes can be used alone, however.
API
WordsCounted
WordsCounted.count(input, options = {})
Tokenises input and initializes a WordsCounted::Counter object with the resulting tokens.
counter = WordsCounted.count("Hello Beirut!")
Accepts two options: exclude and regexp. See Excluding tokens from the analyser and Passing in a custom regexp respectively.
WordsCounted.from_file(path, options = {})
Reads and tokenises a file, and initializes a WordsCounted::Counter object with the resulting tokens.
counter = WordsCounted.from_file("hello_beirut.txt")
Accepts the same options as .count.
Tokeniser
The tokeniser allows you to tokenise text in a variety of ways. You can pass in your own rules for tokenisation, and apply a powerful filter with any combination of rules as long as they can boil down into a lambda.
Out of the box the tokeniser includes only alpha chars. Hyphenated tokens and tokens with apostrophes are considered a single token.
#tokenise([pattern: TOKEN_REGEXP, exclude: nil])
tokeniser = WordsCounted::Tokeniser.new("Hello Beirut!").tokenise
# With `exclude`
tokeniser = WordsCounted::Tokeniser.new("Hello Beirut!").tokenise(exclude: "hello")
# With `pattern`
tokeniser = WordsCounted::Tokeniser.new("I <3 Beirut!").tokenise(pattern: /[a-z]/i)
See Excluding tokens from the analyser and Passing in a custom regexp for more information.
Counter
The WordsCounted::Counter class allows you to collect various statistics from an array of tokens.
#token_count
Returns the token count of a given string.
counter.token_count #=> 15
#token_frequency
Returns a sorted (unstable) two-dimensional array where each element is a token and its frequency. The array is sorted by frequency in descending order.
counter.token_frequency
[
["the", 2],
["are", 2],
["we", 1],
# ...
["all", 1]
]
#most_frequent_tokens
Returns a hash where each key-value pair is a token and its frequency.
counter.most_frequent_tokens
{ "are" => 2, "the" => 2 }
#token_lengths
Returns a sorted (unstable) two-dimentional array where each element contains a token and its length. The array is sorted by length in descending order.
counter.token_lengths
[
["looking", 7],
["gutter", 6],
["stars", 5],
# ...
["in", 2]
]
#longest_tokens
Returns a hash where each key-value pair is a token and its length.
counter.longest_tokens
{ "looking" => 7 }
#token_density([ precision: 2 ])
Returns a sorted (unstable) two-dimentional array where each element contains a token and its density as a float, rounded to a precision of two. The array is sorted by density in descending order. It accepts a precision argument, which must be a float.
counter.token_density
[
["are", 0.13],
["the", 0.13],
["but", 0.07 ],
# ...
["we", 0.07 ]
]
#char_count
Returns the char count of tokens.
counter.char_count #=> 76
#average_chars_per_token([ precision: 2 ])
Returns the average char count per token rounded to two decimal places. Accepts a precision argument which defaults to two. Precision must be a float.
counter.average_chars_per_token #=> 4
#uniq_token_count
Returns the number of unique tokens.
counter.uniq_token_count #=> 13
Excluding tokens from the tokeniser
You can exclude anything you want from the input by passing the exclude option. The exclude option accepts a variety of filters and is extremely flexible.
- A space-delimited string. The filter will normalise the string.
- A regular expression.
- A lambda.
- A symbol that names a predicate method. For example
:odd?. - An array of any combination of the above.
tokeniser =
WordsCounted::Tokeniser.new(
"Magnificent! That was magnificent, Trevor."
)
# Using a string
tokeniser.tokenise(exclude: "was magnificent")
# => ["that", "trevor"]
# Using a regular expression
tokeniser.tokenise(exclude: /trevor/)
# => ["magnificent", "that", "was", "magnificent"]
# Using a lambda
tokeniser.tokenise(exclude: ->(t) { t.length < 4 })
# => ["magnificent", "that", "magnificent", "trevor"]
# Using symbol
tokeniser = WordsCounted::Tokeniser.new("Hello! محمد")
tokeniser.tokenise(exclude: :ascii_only?)
# => ["محمد"]
# Using an array
tokeniser = WordsCounted::Tokeniser.new(
"Hello! اسماءنا هي محمد، كارولينا، سامي، وداني"
)
tokeniser.tokenise(
exclude: [:ascii_only?, /محمد/, ->(t) { t.length > 6}, "و"]
)
# => ["هي", "سامي", "وداني"]
Passing in a custom regexp
The default regexp accounts for letters, hyphenated tokens, and apostrophes. This means twenty-one is treated as one token. So is Mohamad's.
/[\p{Alpha}\-']+/
You can pass your own criteria as a Ruby regular expression to split your string as desired.
For example, if you wanted to include numbers, you can override the regular expression:
counter = WordsCounted.count("Numbers 1, 2, and 3", pattern: /[\p{Alnum}\-']+/)
counter.tokens
#=> ["numbers", "1", "2", "and", "3"]
Opening and reading files
Use the from_file method to open files. from_file accepts the same options as .count. The file path can be a URL.
counter = WordsCounted.from_file("url/or/path/to/file.text")
Gotchas
A hyphen used in leu of an em or en dash will form part of the token. This affects the tokeniser algorithm.
counter = WordsCounted.count("How do you do?-you are well, I see.")
counter.token_frequency
[
["do", 2],
["how", 1],
["you", 1],
["-you", 1], # WTF, mate!
["are", 1],
# ...
]
In this example -you and you are separate tokens. Also, the tokeniser does not include numbers by default. Remember that you can pass your own regular expression if the default behaviour does not fit your needs.
A note on case sensitivity
The program will normalise (downcase) all incoming strings for consistency and filters.
Roadmap
Ability to open URLs
def self.from_url
# open url and send string here after removing html
end
Are you using WordsCounted to do something interesting? Please tell me about it.
Demo
Visit this website for one example of what you can do with WordsCounted.
Contributors
See contributors.
Contributing
- Fork it
- Create your feature branch (
git checkout -b my-new-feature) - Commit your changes (
git commit -am 'Add some feature') - Push to the branch (
git push origin my-new-feature) - Create new Pull Request
Author: Abitdodgy
Source Code: https://github.com/abitdodgy/words_counted
License: MIT license
1624658400
PAID NETWORK Review, Is it worth Investing in? Token Sale Coming Soon !!
Hey guys, in this video I review PAID NETWORK. This is a DeFi project that aims to solve complex legal process using decentralised protocols and DeFi products for 2021.
PAID Network is an ecosystem DAPP that leverages blockchain technology to deliver DeFi powered SMART Agreements to make business exponentially more efficient. We allow users to create their own policy, to ensure they Get PAID.
📺 The video in this post was made by Crypto expat
The origin of the article: https://www.youtube.com/watch?v=ZIU5javfL90
🔺 DISCLAIMER: The article is for information sharing. The content of this video is solely the opinions of the speaker who is not a licensed financial advisor or registered investment advisor. Not investment advice or legal advice.
Cryptocurrency trading is VERY risky. Make sure you understand these risks and that you are responsible for what you do with your money
🔥 If you’re a beginner. I believe the article below will be useful to you ☞ What You Should Know Before Investing in Cryptocurrency - For Beginner
⭐ ⭐ ⭐The project is of interest to the community. Join to Get free ‘GEEK coin’ (GEEKCASH coin)!
☞ **-----CLICK HERE-----**⭐ ⭐ ⭐
Thanks for visiting and watching! Please don’t forget to leave a like, comment and share!
#bitcoin #blockchain #paid network #paid network review #token sale #paid network review, is it worth investing in? token sale coming soon !!
1622197808
SafeMoon Clone | Create A DeFi Token Like SafeMoon | DeFi token like SafeMoon
SafeMoon is a decentralized finance (DeFi) token. This token consists of RFI tokenomics and auto-liquidity generating protocol. A DeFi token like SafeMoon has reached the mainstream standards under the Binance Smart Chain. Its success and popularity have been immense, thus, making the majority of the business firms adopt this style of cryptocurrency as an alternative.
A DeFi token like SafeMoon is almost similar to the other crypto-token, but the only difference being that it charges a 10% transaction fee from the users who sell their tokens, in which 5% of the fee is distributed to the remaining SafeMoon owners. This feature rewards the owners for holding onto their tokens.
Read More @ https://bit.ly/3oFbJoJ
#create a defi token like safemoon #defi token like safemoon #safemoon token #safemoon token clone #defi token