1616036577
What is Dora Factory (DORA) | What is Dora Factory token | What is DORA token
As you may know, Dorayaki is the favorite snack of Doraemon. It’s a dessert with red bean filling between two slices of sweet fluffy pancakes.
Dora Factory producesDorayaki, and Dorayaki is the fuel of all DAO ventures that perform on-chain governance.
Dora Factory is governed by DORA holders. DORA will be the utility token that binds the network together. Users and holders of DORA can stake, mine and pay DORA tokens. Developers who build will be rewarded with DORA tokens.
The initial total supply of DORA is 10,000,000.
Before we get to the distribution details, let’s walk through the utility of DORA token in the network:
DAO Staking
Every DAO is created by staking 100 DORA tokens on-chain. When the DAO is terminated, staked DORA tokens will be released. Later, when the number of DAOs grows and the value of DORA changes, a fixed staking price might be necessary when a DAO is created. This can be achieved by implementing a price oracle with an off-chain worker, or simply using an existing oracle service provided by some other parachain.
Validator Staking, Mining and Inflation
Dora Factory will first be a POA network, and transfer into an NPoS network when deployed on Kusama. Validators and nominators maintain the parachain by verifying transactions and producing blocks. As a result of that, validators and nominators are rewarded with tokens from inflation. Besides inflation, transaction fees will be distributed to validators and nominators, network fees (tax) collected from users and DAOs will be paid proportionally across all addresses that staked DORAs.
The inflation is 1,000,000 DORAs per year until a super majority of the community votes otherwise. Therefore, unlike Polkadot itself, this is a linear inflation starting from 10% in the first year. Inflation rate is decreasing every year. Among the tokens generated from inflation every year, 2% will go to a Dora Factory Treasury, the rest of DORAs generated from inflation will be distributed to the nominators and validators.
Network Fees (Tax)
Pallets can charge fees. DAOs will have to “burn” DORA tokens (or pay fees) to use pallets (there can be free pallets!). Network Fees paid by DAOs are taxed by Dora Factory. The tax will be proportionally distributed to all staking addresses.
Network Rewards
The network rewards validators, pallet developers and venture builders. Validators are rewarded from inflation as well as tax collected from DAOs. Non-validator DORA stakers are rewarded by network tax. Developers are rewarded by pallet income.
Transaction Fees
A small transaction fee will be imposed on all transactions on the parachain. The Transaction fees collected will be distributed to nominators and validators.
🏡 Governance
Unlike Internet SaaS platforms, Dora Factory is governed by the community of DORA holders instead of a company.
There are two governance bodies. The Governance DAO and the Council. The governance DAO is a DAO of all DORA holders. The council is the core development team that is responsible for the project’s roadmap until sudo is removed.
The Governance DAO is a DAO produced by Dora Factory. After Dora Factory’s mainnet launch, the governance DAO will be presented on Dora Factory’s client frontend. All DORA holders are members of the DAO, and all staking addresses can vote on issues (we call them “DORA Voters”). There are several situations where governance comes into play.
Approve Major Parachain Mainnet Upgrade
Any major upgrade of the parachain mainnet (parachain consensus, governance rules, mining & staking rules) needs a ⅔ majority vote from DORA Voters.
Approve Community Grants
There will be ongoing quadratic funding grant rounds to support new ideas to improve the infrastructure and creating new pallets on Dora Factory. Community can donate to projects they deem important with DORA and a matching fund will be distributed to the projects based on the quadratic funding scheme. The community grant will be hosted on HackerLink.
Every community quadratic funding grant needs to be approved by a simple majority vote from DORA Voters.
In addition to the governance DAO, Dora Factory has a Council, which is responsible for implementing the roadmap and executing key operations.
The Council also has power to make minor changes and upgrades to the infrastructure, and add pallets based on the procedure of adding pallets.
Adding Pallets
Pallets can be developed either by Dora Factory developers or any community developer (group) from any place. Mobilizing the developer community to invent DAO pallets is critical to Dora Factory. On Polkadot, pallets can be developed and tested freely on local substrate nodes. When a pallet is added to the parachain, its security must be ensured.
A pallet (either developed by Dora Factory core dev team, or by community developers) needs to go through a thorough testing and a security audit before it can be submitted to the Dora Factory testnet. The pallet will reside on the testnet for a certain period of time before it can be compiled into Dora Factory’s Polkadot Parachain. This process is facilitated by the council.
Open Grants
The council also controls an open grant which supports new ideas, schemes and algorithms of on-chain governance, essential DAO pallets, important toolkits. In order to make a decision to issue an open grant, a simple majority must be reached among the councillors.
Bounties
There will be certain features and improvements proposed by users or core developers. An open bounty program will be hosted on HackerLink. Both core DORAs and community projects can issue a bounty. Every bounty is linked to a GitHub issue so that workflows can be managed on GitHub. Many types of cryptos can be used for bounty payments.
Smaller features and improvements that are not within the scope of open grants might be posted as bounties.
Sudo and Election
The councillors will be appointed by Dora Factory Foundation until Sudo is removed. When Sudo is removed, councillors will be elected by DORA Voters. The rules of election will be then hard coded into a governance pallet and be added to the Governance DAO. At that time, Dora Factory will enter DAO governance.
Token Distribution
Dora Factory’s native token is DORA. DORA empowers the DAO-as-a-Service infrastructure. There will be two stages of DORA token.
**In the first stage, Dora Factory will issue a standard ERC20 token, DORA. **The ERC20 DORA token will be used in various Dora Factory products. The total supply of ERC20 DORA token will be fixed.
After the Dora Factory mainnet is launched, ERC20 DORA tokens will be converted to the mainnet native token, and it enters the second stage. In this stage, network staking and network rewards will start, and there will be a linear inflation of 1,000,000 DORA tokens every year, which will be used to reward network validators and community developer grants. (note: this will only happen after mainnet is launched).
The supply and allocation plan for DORA is designed as such:
- Initial total supply: 10,000,000 DORA
- The distribution plan for the tokens is as shown in the tables below.
Dorayaki Token Distribution Form
There will be roughly 18% of DORA liquidated into secondary market after the Token Generation Event.
Dorayaki Token Distribution Chart
The above token distribution plan may be subject to changes via community governance in the future.
Would you like to earn many tokens and cryptocurrencies right now! ☞ CLICK HERE
How and Where to Buy DORA ?
You will have to first buy one of the major cryptocurrencies, usually either Bitcoin (BTC), Ethereum (ETH), Tether (USDT), BNB, BUSD …
We will use Binance Exchange here as it is one of the largest crypto exchanges that accept fiat deposits.
Binance is a popular cryptocurrency exchange which was started in China but then moved their headquarters to the crypto-friendly Island of Malta in the EU. Binance is popular for its crypto to crypto exchange services. Binance exploded onto the scene in the mania of 2017 and has since gone on to become the top crypto exchange in the world.
Once you finished the KYC process. You will be asked to add a payment method. Here you can either choose to provide a credit/debit card or use a bank transfer, and buy one of the major cryptocurrencies, usually either Bitcoin (BTC), Ethereum (ETH), Tether (USDT), BNB, BUSD
Step by Step Guide : What is Binance | How to Create an account on Binance (Updated 2021)
After the deposit is confirmed you may then purchase DORA from Website: https://hackerlink.io/en/Grant?type=HECO
Apart from the exchange(s) above, there are a few popular crypto exchanges where they have decent daily trading volumes and a huge user base. This will ensure you will be able to sell your coins at any time and the fees will usually be lower. It is suggested that you also register on these exchanges since once DORA gets listed there it will attract a large amount of trading volumes from the users there, that means you will be having some great trading opportunities!
Top exchanges for token-coin trading. Follow instructions and make unlimited money
☞ https://www.binance.com
☞ https://www.bittrex.com
☞ https://www.poloniex.com
☞ https://www.bitfinex.com
☞ https://www.huobi.com
☞ https://www.mxc.ai
☞ https://www.probit.com
☞ https://www.gate.io
☞ https://www.coinbase.com
Find more information DORA
☞ Website ☞ Announcement ☞ Social Channel ☞ Social Channel 2 ☞ Coinmarketcap
🔺DISCLAIMER: Trading Cryptocurrency is VERY risky. Make sure that you understand these risks if you are a beginner. The Information in the post is my OPINION and not financial advice. You are responsible for what you do with your funds
Learn about Cryptocurrency in this article ☞ What You Should Know Before Investing in Cryptocurrency - For Beginner
I hope this post will help you. If you liked this, please sharing it with others. Thank you!
#blockchain #bitcoin #dora factory #dora
What is GEEK
Buddha Community
1616036577
What is Dora Factory (DORA) | What is Dora Factory token | What is DORA token
As you may know, Dorayaki is the favorite snack of Doraemon. It’s a dessert with red bean filling between two slices of sweet fluffy pancakes.
Dora Factory producesDorayaki, and Dorayaki is the fuel of all DAO ventures that perform on-chain governance.
Dora Factory is governed by DORA holders. DORA will be the utility token that binds the network together. Users and holders of DORA can stake, mine and pay DORA tokens. Developers who build will be rewarded with DORA tokens.
The initial total supply of DORA is 10,000,000.
Before we get to the distribution details, let’s walk through the utility of DORA token in the network:
DAO Staking
Every DAO is created by staking 100 DORA tokens on-chain. When the DAO is terminated, staked DORA tokens will be released. Later, when the number of DAOs grows and the value of DORA changes, a fixed staking price might be necessary when a DAO is created. This can be achieved by implementing a price oracle with an off-chain worker, or simply using an existing oracle service provided by some other parachain.
Validator Staking, Mining and Inflation
Dora Factory will first be a POA network, and transfer into an NPoS network when deployed on Kusama. Validators and nominators maintain the parachain by verifying transactions and producing blocks. As a result of that, validators and nominators are rewarded with tokens from inflation. Besides inflation, transaction fees will be distributed to validators and nominators, network fees (tax) collected from users and DAOs will be paid proportionally across all addresses that staked DORAs.
The inflation is 1,000,000 DORAs per year until a super majority of the community votes otherwise. Therefore, unlike Polkadot itself, this is a linear inflation starting from 10% in the first year. Inflation rate is decreasing every year. Among the tokens generated from inflation every year, 2% will go to a Dora Factory Treasury, the rest of DORAs generated from inflation will be distributed to the nominators and validators.
Network Fees (Tax)
Pallets can charge fees. DAOs will have to “burn” DORA tokens (or pay fees) to use pallets (there can be free pallets!). Network Fees paid by DAOs are taxed by Dora Factory. The tax will be proportionally distributed to all staking addresses.
Network Rewards
The network rewards validators, pallet developers and venture builders. Validators are rewarded from inflation as well as tax collected from DAOs. Non-validator DORA stakers are rewarded by network tax. Developers are rewarded by pallet income.
Transaction Fees
A small transaction fee will be imposed on all transactions on the parachain. The Transaction fees collected will be distributed to nominators and validators.
🏡 Governance
Unlike Internet SaaS platforms, Dora Factory is governed by the community of DORA holders instead of a company.
There are two governance bodies. The Governance DAO and the Council. The governance DAO is a DAO of all DORA holders. The council is the core development team that is responsible for the project’s roadmap until sudo is removed.
The Governance DAO is a DAO produced by Dora Factory. After Dora Factory’s mainnet launch, the governance DAO will be presented on Dora Factory’s client frontend. All DORA holders are members of the DAO, and all staking addresses can vote on issues (we call them “DORA Voters”). There are several situations where governance comes into play.
Approve Major Parachain Mainnet Upgrade
Any major upgrade of the parachain mainnet (parachain consensus, governance rules, mining & staking rules) needs a ⅔ majority vote from DORA Voters.
Approve Community Grants
There will be ongoing quadratic funding grant rounds to support new ideas to improve the infrastructure and creating new pallets on Dora Factory. Community can donate to projects they deem important with DORA and a matching fund will be distributed to the projects based on the quadratic funding scheme. The community grant will be hosted on HackerLink.
Every community quadratic funding grant needs to be approved by a simple majority vote from DORA Voters.
In addition to the governance DAO, Dora Factory has a Council, which is responsible for implementing the roadmap and executing key operations.
The Council also has power to make minor changes and upgrades to the infrastructure, and add pallets based on the procedure of adding pallets.
Adding Pallets
Pallets can be developed either by Dora Factory developers or any community developer (group) from any place. Mobilizing the developer community to invent DAO pallets is critical to Dora Factory. On Polkadot, pallets can be developed and tested freely on local substrate nodes. When a pallet is added to the parachain, its security must be ensured.
A pallet (either developed by Dora Factory core dev team, or by community developers) needs to go through a thorough testing and a security audit before it can be submitted to the Dora Factory testnet. The pallet will reside on the testnet for a certain period of time before it can be compiled into Dora Factory’s Polkadot Parachain. This process is facilitated by the council.
Open Grants
The council also controls an open grant which supports new ideas, schemes and algorithms of on-chain governance, essential DAO pallets, important toolkits. In order to make a decision to issue an open grant, a simple majority must be reached among the councillors.
Bounties
There will be certain features and improvements proposed by users or core developers. An open bounty program will be hosted on HackerLink. Both core DORAs and community projects can issue a bounty. Every bounty is linked to a GitHub issue so that workflows can be managed on GitHub. Many types of cryptos can be used for bounty payments.
Smaller features and improvements that are not within the scope of open grants might be posted as bounties.
Sudo and Election
The councillors will be appointed by Dora Factory Foundation until Sudo is removed. When Sudo is removed, councillors will be elected by DORA Voters. The rules of election will be then hard coded into a governance pallet and be added to the Governance DAO. At that time, Dora Factory will enter DAO governance.
Token Distribution
Dora Factory’s native token is DORA. DORA empowers the DAO-as-a-Service infrastructure. There will be two stages of DORA token.
**In the first stage, Dora Factory will issue a standard ERC20 token, DORA. **The ERC20 DORA token will be used in various Dora Factory products. The total supply of ERC20 DORA token will be fixed.
After the Dora Factory mainnet is launched, ERC20 DORA tokens will be converted to the mainnet native token, and it enters the second stage. In this stage, network staking and network rewards will start, and there will be a linear inflation of 1,000,000 DORA tokens every year, which will be used to reward network validators and community developer grants. (note: this will only happen after mainnet is launched).
The supply and allocation plan for DORA is designed as such:
- Initial total supply: 10,000,000 DORA
- The distribution plan for the tokens is as shown in the tables below.
Dorayaki Token Distribution Form
There will be roughly 18% of DORA liquidated into secondary market after the Token Generation Event.
Dorayaki Token Distribution Chart
The above token distribution plan may be subject to changes via community governance in the future.
Would you like to earn many tokens and cryptocurrencies right now! ☞ CLICK HERE
How and Where to Buy DORA ?
You will have to first buy one of the major cryptocurrencies, usually either Bitcoin (BTC), Ethereum (ETH), Tether (USDT), BNB, BUSD …
We will use Binance Exchange here as it is one of the largest crypto exchanges that accept fiat deposits.
Binance is a popular cryptocurrency exchange which was started in China but then moved their headquarters to the crypto-friendly Island of Malta in the EU. Binance is popular for its crypto to crypto exchange services. Binance exploded onto the scene in the mania of 2017 and has since gone on to become the top crypto exchange in the world.
Once you finished the KYC process. You will be asked to add a payment method. Here you can either choose to provide a credit/debit card or use a bank transfer, and buy one of the major cryptocurrencies, usually either Bitcoin (BTC), Ethereum (ETH), Tether (USDT), BNB, BUSD
Step by Step Guide : What is Binance | How to Create an account on Binance (Updated 2021)
After the deposit is confirmed you may then purchase DORA from Website: https://hackerlink.io/en/Grant?type=HECO
Apart from the exchange(s) above, there are a few popular crypto exchanges where they have decent daily trading volumes and a huge user base. This will ensure you will be able to sell your coins at any time and the fees will usually be lower. It is suggested that you also register on these exchanges since once DORA gets listed there it will attract a large amount of trading volumes from the users there, that means you will be having some great trading opportunities!
Top exchanges for token-coin trading. Follow instructions and make unlimited money
☞ https://www.binance.com
☞ https://www.bittrex.com
☞ https://www.poloniex.com
☞ https://www.bitfinex.com
☞ https://www.huobi.com
☞ https://www.mxc.ai
☞ https://www.probit.com
☞ https://www.gate.io
☞ https://www.coinbase.com
Find more information DORA
☞ Website ☞ Announcement ☞ Social Channel ☞ Social Channel 2 ☞ Coinmarketcap
🔺DISCLAIMER: Trading Cryptocurrency is VERY risky. Make sure that you understand these risks if you are a beginner. The Information in the post is my OPINION and not financial advice. You are responsible for what you do with your funds
Learn about Cryptocurrency in this article ☞ What You Should Know Before Investing in Cryptocurrency - For Beginner
I hope this post will help you. If you liked this, please sharing it with others. Thank you!
#blockchain #bitcoin #dora factory #dora
1658068560
WordsCounted: A Ruby Natural Language Processor
WordsCounted
We are all in the gutter, but some of us are looking at the stars.
-- Oscar Wilde
WordsCounted is a Ruby NLP (natural language processor). WordsCounted lets you implement powerful tokensation strategies with a very flexible tokeniser class.
Features
- Out of the box, get the following data from any string or readable file, or URL:
- Token count and unique token count
- Token densities, frequencies, and lengths
- Char count and average chars per token
- The longest tokens and their lengths
- The most frequent tokens and their frequencies.
- A flexible way to exclude tokens from the tokeniser. You can pass a string, regexp, symbol, lambda, or an array of any combination of those types for powerful tokenisation strategies.
- Pass your own regexp rules to the tokeniser if you prefer. The default regexp filters special characters but keeps hyphens and apostrophes. It also plays nicely with diacritics (UTF and unicode characters): Bayrūt is treated as
["Bayrūt"]and not["Bayr", "ū", "t"], for example. - Opens and reads files. Pass in a file path or a url instead of a string.
Installation
Add this line to your application's Gemfile:
gem 'words_counted'
And then execute:
$ bundle
Or install it yourself as:
$ gem install words_counted
Usage
Pass in a string or a file path, and an optional filter and/or regexp.
counter = WordsCounted.count(
"We are all in the gutter, but some of us are looking at the stars."
)
# Using a file
counter = WordsCounted.from_file("path/or/url/to/my/file.txt")
.count and .from_file are convenience methods that take an input, tokenise it, and return an instance of WordsCounted::Counter initialized with the tokens. The WordsCounted::Tokeniser and WordsCounted::Counter classes can be used alone, however.
API
WordsCounted
WordsCounted.count(input, options = {})
Tokenises input and initializes a WordsCounted::Counter object with the resulting tokens.
counter = WordsCounted.count("Hello Beirut!")
Accepts two options: exclude and regexp. See Excluding tokens from the analyser and Passing in a custom regexp respectively.
WordsCounted.from_file(path, options = {})
Reads and tokenises a file, and initializes a WordsCounted::Counter object with the resulting tokens.
counter = WordsCounted.from_file("hello_beirut.txt")
Accepts the same options as .count.
Tokeniser
The tokeniser allows you to tokenise text in a variety of ways. You can pass in your own rules for tokenisation, and apply a powerful filter with any combination of rules as long as they can boil down into a lambda.
Out of the box the tokeniser includes only alpha chars. Hyphenated tokens and tokens with apostrophes are considered a single token.
#tokenise([pattern: TOKEN_REGEXP, exclude: nil])
tokeniser = WordsCounted::Tokeniser.new("Hello Beirut!").tokenise
# With `exclude`
tokeniser = WordsCounted::Tokeniser.new("Hello Beirut!").tokenise(exclude: "hello")
# With `pattern`
tokeniser = WordsCounted::Tokeniser.new("I <3 Beirut!").tokenise(pattern: /[a-z]/i)
See Excluding tokens from the analyser and Passing in a custom regexp for more information.
Counter
The WordsCounted::Counter class allows you to collect various statistics from an array of tokens.
#token_count
Returns the token count of a given string.
counter.token_count #=> 15
#token_frequency
Returns a sorted (unstable) two-dimensional array where each element is a token and its frequency. The array is sorted by frequency in descending order.
counter.token_frequency
[
["the", 2],
["are", 2],
["we", 1],
# ...
["all", 1]
]
#most_frequent_tokens
Returns a hash where each key-value pair is a token and its frequency.
counter.most_frequent_tokens
{ "are" => 2, "the" => 2 }
#token_lengths
Returns a sorted (unstable) two-dimentional array where each element contains a token and its length. The array is sorted by length in descending order.
counter.token_lengths
[
["looking", 7],
["gutter", 6],
["stars", 5],
# ...
["in", 2]
]
#longest_tokens
Returns a hash where each key-value pair is a token and its length.
counter.longest_tokens
{ "looking" => 7 }
#token_density([ precision: 2 ])
Returns a sorted (unstable) two-dimentional array where each element contains a token and its density as a float, rounded to a precision of two. The array is sorted by density in descending order. It accepts a precision argument, which must be a float.
counter.token_density
[
["are", 0.13],
["the", 0.13],
["but", 0.07 ],
# ...
["we", 0.07 ]
]
#char_count
Returns the char count of tokens.
counter.char_count #=> 76
#average_chars_per_token([ precision: 2 ])
Returns the average char count per token rounded to two decimal places. Accepts a precision argument which defaults to two. Precision must be a float.
counter.average_chars_per_token #=> 4
#uniq_token_count
Returns the number of unique tokens.
counter.uniq_token_count #=> 13
Excluding tokens from the tokeniser
You can exclude anything you want from the input by passing the exclude option. The exclude option accepts a variety of filters and is extremely flexible.
- A space-delimited string. The filter will normalise the string.
- A regular expression.
- A lambda.
- A symbol that names a predicate method. For example
:odd?. - An array of any combination of the above.
tokeniser =
WordsCounted::Tokeniser.new(
"Magnificent! That was magnificent, Trevor."
)
# Using a string
tokeniser.tokenise(exclude: "was magnificent")
# => ["that", "trevor"]
# Using a regular expression
tokeniser.tokenise(exclude: /trevor/)
# => ["magnificent", "that", "was", "magnificent"]
# Using a lambda
tokeniser.tokenise(exclude: ->(t) { t.length < 4 })
# => ["magnificent", "that", "magnificent", "trevor"]
# Using symbol
tokeniser = WordsCounted::Tokeniser.new("Hello! محمد")
tokeniser.tokenise(exclude: :ascii_only?)
# => ["محمد"]
# Using an array
tokeniser = WordsCounted::Tokeniser.new(
"Hello! اسماءنا هي محمد، كارولينا، سامي، وداني"
)
tokeniser.tokenise(
exclude: [:ascii_only?, /محمد/, ->(t) { t.length > 6}, "و"]
)
# => ["هي", "سامي", "وداني"]
Passing in a custom regexp
The default regexp accounts for letters, hyphenated tokens, and apostrophes. This means twenty-one is treated as one token. So is Mohamad's.
/[\p{Alpha}\-']+/
You can pass your own criteria as a Ruby regular expression to split your string as desired.
For example, if you wanted to include numbers, you can override the regular expression:
counter = WordsCounted.count("Numbers 1, 2, and 3", pattern: /[\p{Alnum}\-']+/)
counter.tokens
#=> ["numbers", "1", "2", "and", "3"]
Opening and reading files
Use the from_file method to open files. from_file accepts the same options as .count. The file path can be a URL.
counter = WordsCounted.from_file("url/or/path/to/file.text")
Gotchas
A hyphen used in leu of an em or en dash will form part of the token. This affects the tokeniser algorithm.
counter = WordsCounted.count("How do you do?-you are well, I see.")
counter.token_frequency
[
["do", 2],
["how", 1],
["you", 1],
["-you", 1], # WTF, mate!
["are", 1],
# ...
]
In this example -you and you are separate tokens. Also, the tokeniser does not include numbers by default. Remember that you can pass your own regular expression if the default behaviour does not fit your needs.
A note on case sensitivity
The program will normalise (downcase) all incoming strings for consistency and filters.
Roadmap
Ability to open URLs
def self.from_url
# open url and send string here after removing html
end
Are you using WordsCounted to do something interesting? Please tell me about it.
Demo
Visit this website for one example of what you can do with WordsCounted.
Contributors
See contributors.
Contributing
- Fork it
- Create your feature branch (
git checkout -b my-new-feature) - Commit your changes (
git commit -am 'Add some feature') - Push to the branch (
git push origin my-new-feature) - Create new Pull Request
Author: Abitdodgy
Source Code: https://github.com/abitdodgy/words_counted
License: MIT license
1659601560
Words Counted: A Ruby Natural Language Processor.
WordsCounted
We are all in the gutter, but some of us are looking at the stars.
-- Oscar Wilde
WordsCounted is a Ruby NLP (natural language processor). WordsCounted lets you implement powerful tokensation strategies with a very flexible tokeniser class.
Are you using WordsCounted to do something interesting? Please tell me about it.
Demo
Visit this website for one example of what you can do with WordsCounted.
Features
- Out of the box, get the following data from any string or readable file, or URL:
- Token count and unique token count
- Token densities, frequencies, and lengths
- Char count and average chars per token
- The longest tokens and their lengths
- The most frequent tokens and their frequencies.
- A flexible way to exclude tokens from the tokeniser. You can pass a string, regexp, symbol, lambda, or an array of any combination of those types for powerful tokenisation strategies.
- Pass your own regexp rules to the tokeniser if you prefer. The default regexp filters special characters but keeps hyphens and apostrophes. It also plays nicely with diacritics (UTF and unicode characters): Bayrūt is treated as
["Bayrūt"]and not["Bayr", "ū", "t"], for example. - Opens and reads files. Pass in a file path or a url instead of a string.
Installation
Add this line to your application's Gemfile:
gem 'words_counted'
And then execute:
$ bundle
Or install it yourself as:
$ gem install words_counted
Usage
Pass in a string or a file path, and an optional filter and/or regexp.
counter = WordsCounted.count(
"We are all in the gutter, but some of us are looking at the stars."
)
# Using a file
counter = WordsCounted.from_file("path/or/url/to/my/file.txt")
.count and .from_file are convenience methods that take an input, tokenise it, and return an instance of WordsCounted::Counter initialized with the tokens. The WordsCounted::Tokeniser and WordsCounted::Counter classes can be used alone, however.
API
WordsCounted
WordsCounted.count(input, options = {})
Tokenises input and initializes a WordsCounted::Counter object with the resulting tokens.
counter = WordsCounted.count("Hello Beirut!")
Accepts two options: exclude and regexp. See Excluding tokens from the analyser and Passing in a custom regexp respectively.
WordsCounted.from_file(path, options = {})
Reads and tokenises a file, and initializes a WordsCounted::Counter object with the resulting tokens.
counter = WordsCounted.from_file("hello_beirut.txt")
Accepts the same options as .count.
Tokeniser
The tokeniser allows you to tokenise text in a variety of ways. You can pass in your own rules for tokenisation, and apply a powerful filter with any combination of rules as long as they can boil down into a lambda.
Out of the box the tokeniser includes only alpha chars. Hyphenated tokens and tokens with apostrophes are considered a single token.
#tokenise([pattern: TOKEN_REGEXP, exclude: nil])
tokeniser = WordsCounted::Tokeniser.new("Hello Beirut!").tokenise
# With `exclude`
tokeniser = WordsCounted::Tokeniser.new("Hello Beirut!").tokenise(exclude: "hello")
# With `pattern`
tokeniser = WordsCounted::Tokeniser.new("I <3 Beirut!").tokenise(pattern: /[a-z]/i)
See Excluding tokens from the analyser and Passing in a custom regexp for more information.
Counter
The WordsCounted::Counter class allows you to collect various statistics from an array of tokens.
#token_count
Returns the token count of a given string.
counter.token_count #=> 15
#token_frequency
Returns a sorted (unstable) two-dimensional array where each element is a token and its frequency. The array is sorted by frequency in descending order.
counter.token_frequency
[
["the", 2],
["are", 2],
["we", 1],
# ...
["all", 1]
]
#most_frequent_tokens
Returns a hash where each key-value pair is a token and its frequency.
counter.most_frequent_tokens
{ "are" => 2, "the" => 2 }
#token_lengths
Returns a sorted (unstable) two-dimentional array where each element contains a token and its length. The array is sorted by length in descending order.
counter.token_lengths
[
["looking", 7],
["gutter", 6],
["stars", 5],
# ...
["in", 2]
]
#longest_tokens
Returns a hash where each key-value pair is a token and its length.
counter.longest_tokens
{ "looking" => 7 }
#token_density([ precision: 2 ])
Returns a sorted (unstable) two-dimentional array where each element contains a token and its density as a float, rounded to a precision of two. The array is sorted by density in descending order. It accepts a precision argument, which must be a float.
counter.token_density
[
["are", 0.13],
["the", 0.13],
["but", 0.07 ],
# ...
["we", 0.07 ]
]
#char_count
Returns the char count of tokens.
counter.char_count #=> 76
#average_chars_per_token([ precision: 2 ])
Returns the average char count per token rounded to two decimal places. Accepts a precision argument which defaults to two. Precision must be a float.
counter.average_chars_per_token #=> 4
#uniq_token_count
Returns the number of unique tokens.
counter.uniq_token_count #=> 13
Excluding tokens from the tokeniser
You can exclude anything you want from the input by passing the exclude option. The exclude option accepts a variety of filters and is extremely flexible.
- A space-delimited string. The filter will normalise the string.
- A regular expression.
- A lambda.
- A symbol that names a predicate method. For example
:odd?. - An array of any combination of the above.
tokeniser =
WordsCounted::Tokeniser.new(
"Magnificent! That was magnificent, Trevor."
)
# Using a string
tokeniser.tokenise(exclude: "was magnificent")
# => ["that", "trevor"]
# Using a regular expression
tokeniser.tokenise(exclude: /trevor/)
# => ["magnificent", "that", "was", "magnificent"]
# Using a lambda
tokeniser.tokenise(exclude: ->(t) { t.length < 4 })
# => ["magnificent", "that", "magnificent", "trevor"]
# Using symbol
tokeniser = WordsCounted::Tokeniser.new("Hello! محمد")
tokeniser.tokenise(exclude: :ascii_only?)
# => ["محمد"]
# Using an array
tokeniser = WordsCounted::Tokeniser.new(
"Hello! اسماءنا هي محمد، كارولينا، سامي، وداني"
)
tokeniser.tokenise(
exclude: [:ascii_only?, /محمد/, ->(t) { t.length > 6}, "و"]
)
# => ["هي", "سامي", "وداني"]
Passing in a custom regexp
The default regexp accounts for letters, hyphenated tokens, and apostrophes. This means twenty-one is treated as one token. So is Mohamad's.
/[\p{Alpha}\-']+/
You can pass your own criteria as a Ruby regular expression to split your string as desired.
For example, if you wanted to include numbers, you can override the regular expression:
counter = WordsCounted.count("Numbers 1, 2, and 3", pattern: /[\p{Alnum}\-']+/)
counter.tokens
#=> ["numbers", "1", "2", "and", "3"]
Opening and reading files
Use the from_file method to open files. from_file accepts the same options as .count. The file path can be a URL.
counter = WordsCounted.from_file("url/or/path/to/file.text")
Gotchas
A hyphen used in leu of an em or en dash will form part of the token. This affects the tokeniser algorithm.
counter = WordsCounted.count("How do you do?-you are well, I see.")
counter.token_frequency
[
["do", 2],
["how", 1],
["you", 1],
["-you", 1], # WTF, mate!
["are", 1],
# ...
]
In this example -you and you are separate tokens. Also, the tokeniser does not include numbers by default. Remember that you can pass your own regular expression if the default behaviour does not fit your needs.
A note on case sensitivity
The program will normalise (downcase) all incoming strings for consistency and filters.
Roadmap
Ability to open URLs
def self.from_url
# open url and send string here after removing html
end
Contributors
See contributors.
Contributing
- Fork it
- Create your feature branch (
git checkout -b my-new-feature) - Commit your changes (
git commit -am 'Add some feature') - Push to the branch (
git push origin my-new-feature) - Create new Pull Request
Author: abitdodgy
Source code: https://github.com/abitdodgy/words_counted
License: MIT license
#ruby #ruby-on-rails
1622197808
SafeMoon Clone | Create A DeFi Token Like SafeMoon | DeFi token like SafeMoon
SafeMoon is a decentralized finance (DeFi) token. This token consists of RFI tokenomics and auto-liquidity generating protocol. A DeFi token like SafeMoon has reached the mainstream standards under the Binance Smart Chain. Its success and popularity have been immense, thus, making the majority of the business firms adopt this style of cryptocurrency as an alternative.
A DeFi token like SafeMoon is almost similar to the other crypto-token, but the only difference being that it charges a 10% transaction fee from the users who sell their tokens, in which 5% of the fee is distributed to the remaining SafeMoon owners. This feature rewards the owners for holding onto their tokens.
Read More @ https://bit.ly/3oFbJoJ
#create a defi token like safemoon #defi token like safemoon #safemoon token #safemoon token clone #defi token
1625034600
How To Buy Defi Factory Token On Uniswap (DEFT Crypto Coin)
In this video I will show you how to buy defi factory token on uniswap. It’s really easy and it will take you less than a minute to do so!
📺 The video in this post was made by How To Explained
️ The origin of the article: https://www.youtube.com/watch?v=zGOOQz2pm8Y
🔺 DISCLAIMER: The article is for information sharing. The content of this video is solely the opinions of the speaker who is not a licensed financial advisor or registered investment advisor. Not investment advice or legal advice.
Cryptocurrency trading is VERY risky. Make sure you understand these risks and that you are responsible for what you do with your money
🔥 If you’re a beginner. I believe the article below will be useful to you ☞ What You Should Know Before Investing in Cryptocurrency - For Beginner
⭐ ⭐ ⭐The project is of interest to the community. Join to Get free ‘GEEK coin’ (GEEKCASH coin)!
⭐ ⭐ ⭐ Join to Get free ‘GEEK coin’ (GEEKCASH coin)! ☞ https://geekcash.org⭐ ⭐ ⭐
Thanks for visiting and watching! Please don’t forget to leave a like, comment and share!
#defi factory token #what is defi factory token? #uniswap #deft crypto coin #bitcoin #blockchain