1616554805
What is FM Gallery (FMG) | What is FM Gallery token | What is FMG token
FM Gallery is a blockchain-based distribution platform for NFT artworks. We strive to empower the world of arts and fashion with innovative art fragments as well as autonomous fan communities. With blockchain technology, an artwork can be divided into several fragments. Each fragment, represented by an NFT, contains a unique art story. By collecting fragments, users are able to complete the artwork and redeem the completed NFT for physical artworks. Collaborating with top artists, galleries and auction houses around the world, we will continue to release physical and virtual crypto artworks on our blockchain.
FMG will be the governance token for the FM Gallery ecosystem and will conform to the ERC20 standard. The total supply will be fixed at 100,000,000 tokens. Being a DAO, we need to design a fair model for distributing FMG tokens to incentivize the community to use the platform actively. The distribution breakdown is as follows:
1. Team — 20%
2. Ecosystem Fund/Community Governance — 15%
3. Private Sale, Seed Investors and Advisors –15%
4. Community — 50%
• Team
Team and Advisors will be allotted 20% of the supply (20 million FMG). 5% of that (1 million FMG) will be vested on TGE and the remaining will be vested linearly over 12 months on a monthly basis.
• Ecosystem Fund
The Ecosystem Fund will be allotted 15% of the supply (15 million FMG). This will be used for funding for the day-to-day operations and other activities to grow the platform like marketing, partnership, smart contract audit, etc.
• Private Sale, Seed Contributors and Advisors
Private Sale and Seed Contributors will be allotted 15% of the supply (15 million FMG). Vesting schedule for each of them are –
1. Private Sale — 8.6% of the total distribution (8600000 FMG). Around 29% (2500000 FMG) will be vested at TGE and the remaining will be vested linearly over 12 months on a monthly basis.
2. Seed Contributors — 5% of the total distribution (5000000 FMG). Around 20% (1000000 FMG) will be vested at TGE and the remaining will be vested linearly over 12 months on a monthly basis.
3. Advisors — 1.4% of the total distribution (1400000 FMG). Around 20% (280000 FMG) will be vested at 1 month after TGE and the remaining will be vested linearly over the next 11 months on a monthly basis.
• Community
The community will be allotted 50% of the supply (50 million FMG). These tokens will be used to incentivize community engagement in different ways. The distribution model will be front-loaded and will be adjusted later based on community vote. There will be no vesting schedule for this part of the supply. There will be four ways for the community (breakdown can be later adjusted based on community vote) to earn the FMG tokens as incentives for participation in the platform:
1. Blind box NFT purchase rewards (15%)
2. Initial DEX Offering (4%)
3. FMG token liquidity mining (31%)
• Initial DEX Offering
Potentially, 300k worth of ETH will be raised for 2 million FMG at price 1 FMG = 0.15 USD. Another 2 million FMG will be added with 300k USD worth ETH to a Uniswap pool at TGE. More details regarding IDO to be published later.
• Blind Box NFT Purchase Rewards
For every blind box NFT fragment purchased on FM Gallery platform, users will receive rebates in the form of FMG tokens. At the end of every epoch, the amount of FMG tokens a user receives will be directly proportionate to that user’s spending on blind boxes.
Based on initial parameters, the platform will rebate a stable 5% of that user’s spending back in the form of distributed FMG tokens. However, this allocation may change when Limit Epoch Distribution applies and triggers a deferred distribution. We will reveal more details, such as how the distribution value is determined, by the platform launch date.
• Liquidity Mining
To incentivize users to provide liquidity, a total of 31 million FMG tokens will be distributed via the “liquidity mining” program over two years. Initially, we will launch an FMG-ETH Uniswap Pool and provide liquidity mining incentives to it. The epoch size for FMG distribution will be one week for all the pools. For users to be eligible to earn FMG tokens, they must stake liquidity until the end of the epoch.
Initial farming parameters will be shared near the TGE. The community FMG distribution parameters will be later adjusted based on community vote once the protocol moves towards being a DAO.
Thank you for your patience. We’ve been working closely with Poolz and Bounce teams, and have obtained the full lists of all IDO participants who completed the subscription. We will be airdropping the $FMG tokens to those addresses in a short while.
We have also launched our official Uniswap pool (smoothly this time), and here’re the details:
FMG Official Token Address
0x2991341D28Eaea277785D20e1d878D478c7bA4C7
Trade on Uniswap
https://info.uniswap.org/pair/0x8815ee66cbb8fd7adaf199cc5564eb2ac52a9c00
Dextools
https://www.dextools.io/app/uniswap/pair-explorer/0x8815ee66cbb8fd7adaf199cc5564eb2ac52a9c00
Would you like to earn TOKEN right now! ☞ CLICK HERE
How and Where to Buy FM Gallery (FMG)?
FMG has been listed on a number of crypto exchanges, unlike other main cryptocurrencies, it cannot be directly purchased with fiats money. However, You can still easily buy this coin by first buying Bitcoin, ETH, USDT from any large exchanges and then transfer to the exchange that offers to trade this coin, in this guide article we will walk you through in detail the steps to buy FMG
You will have to first buy one of the major cryptocurrencies, usually either Bitcoin (BTC), Ethereum (ETH), Tether (USDT)…
We will use Binance Exchange here as it is one of the largest crypto exchanges that accept fiat deposits.
Binance is a popular cryptocurrency exchange which was started in China but then moved their headquarters to the crypto-friendly Island of Malta in the EU. Binance is popular for its crypto to crypto exchange services. Binance exploded onto the scene in the mania of 2017 and has since gone on to become the top crypto exchange in the world.
Once you finished the KYC process. You will be asked to add a payment method. Here you can either choose to provide a credit/debit card or use a bank transfer, and buy one of the major cryptocurrencies, usually either Bitcoin (BTC), Ethereum (ETH), Tether (USDT)
Step by Step Guide : What is Binance | How to Create an account on Binance (Updated 2021)
Next step - Transfer your cryptos to an Altcoin Exchange
Since FMG is an altcoin we need to transfer our coins to an exchange that FMG can be traded. Below is a list of exchanges that offers to trade FMG in various market pairs, head to their websites and register for an account.
Once finished you will then need to make a BTC/ETH/USDT deposit to the exchange from Binance depending on the available market pairs. After the deposit is confirmed you may then purchase FMG from the exchange: Uniswap (V2)
Apart from the exchange(s) above, there are a few popular crypto exchanges where they have decent daily trading volumes and a huge user base. This will ensure you will be able to sell your coins at any time and the fees will usually be lower. It is suggested that you also register on these exchanges since once FMG gets listed there it will attract a large amount of trading volumes from the users there, that means you will be having some great trading opportunities!
Top exchanges for token-coin trading. Follow instructions and make unlimited money
☞ https://www.binance.com
☞ https://www.bittrex.com
☞ https://www.poloniex.com
☞ https://www.bitfinex.com
☞ https://www.huobi.com
☞ https://www.mxc.ai
☞ https://www.probit.com
☞ https://www.gate.io
☞ https://www.coinbase.com
Find more information FMG
☞ Website ☞ Explorer ☞ Social Channel ☞ Social Channel 2 ☞ Message Board ☞ Message Board 2 ☞ Coinmarketcap
🔺DISCLAIMER: Trading Cryptocurrency is VERY risky. Make sure that you understand these risks if you are a beginner. The Information in the post is my OPINION and not financial advice. You are responsible for what you do with your funds
Learn about Cryptocurrency in this article ☞ What You Should Know Before Investing in Cryptocurrency - For Beginner
Thank for visiting and reading this article! I’m highly appreciate your actions! Please share if you liked it!
#blockchain #bitcoin #fm gallery #fmg
What is GEEK
Buddha Community
1616554805
What is FM Gallery (FMG) | What is FM Gallery token | What is FMG token
FM Gallery is a blockchain-based distribution platform for NFT artworks. We strive to empower the world of arts and fashion with innovative art fragments as well as autonomous fan communities. With blockchain technology, an artwork can be divided into several fragments. Each fragment, represented by an NFT, contains a unique art story. By collecting fragments, users are able to complete the artwork and redeem the completed NFT for physical artworks. Collaborating with top artists, galleries and auction houses around the world, we will continue to release physical and virtual crypto artworks on our blockchain.
FMG will be the governance token for the FM Gallery ecosystem and will conform to the ERC20 standard. The total supply will be fixed at 100,000,000 tokens. Being a DAO, we need to design a fair model for distributing FMG tokens to incentivize the community to use the platform actively. The distribution breakdown is as follows:
1. Team — 20%
2. Ecosystem Fund/Community Governance — 15%
3. Private Sale, Seed Investors and Advisors –15%
4. Community — 50%
• Team
Team and Advisors will be allotted 20% of the supply (20 million FMG). 5% of that (1 million FMG) will be vested on TGE and the remaining will be vested linearly over 12 months on a monthly basis.
• Ecosystem Fund
The Ecosystem Fund will be allotted 15% of the supply (15 million FMG). This will be used for funding for the day-to-day operations and other activities to grow the platform like marketing, partnership, smart contract audit, etc.
• Private Sale, Seed Contributors and Advisors
Private Sale and Seed Contributors will be allotted 15% of the supply (15 million FMG). Vesting schedule for each of them are –
1. Private Sale — 8.6% of the total distribution (8600000 FMG). Around 29% (2500000 FMG) will be vested at TGE and the remaining will be vested linearly over 12 months on a monthly basis.
2. Seed Contributors — 5% of the total distribution (5000000 FMG). Around 20% (1000000 FMG) will be vested at TGE and the remaining will be vested linearly over 12 months on a monthly basis.
3. Advisors — 1.4% of the total distribution (1400000 FMG). Around 20% (280000 FMG) will be vested at 1 month after TGE and the remaining will be vested linearly over the next 11 months on a monthly basis.
• Community
The community will be allotted 50% of the supply (50 million FMG). These tokens will be used to incentivize community engagement in different ways. The distribution model will be front-loaded and will be adjusted later based on community vote. There will be no vesting schedule for this part of the supply. There will be four ways for the community (breakdown can be later adjusted based on community vote) to earn the FMG tokens as incentives for participation in the platform:
1. Blind box NFT purchase rewards (15%)
2. Initial DEX Offering (4%)
3. FMG token liquidity mining (31%)
• Initial DEX Offering
Potentially, 300k worth of ETH will be raised for 2 million FMG at price 1 FMG = 0.15 USD. Another 2 million FMG will be added with 300k USD worth ETH to a Uniswap pool at TGE. More details regarding IDO to be published later.
• Blind Box NFT Purchase Rewards
For every blind box NFT fragment purchased on FM Gallery platform, users will receive rebates in the form of FMG tokens. At the end of every epoch, the amount of FMG tokens a user receives will be directly proportionate to that user’s spending on blind boxes.
Based on initial parameters, the platform will rebate a stable 5% of that user’s spending back in the form of distributed FMG tokens. However, this allocation may change when Limit Epoch Distribution applies and triggers a deferred distribution. We will reveal more details, such as how the distribution value is determined, by the platform launch date.
• Liquidity Mining
To incentivize users to provide liquidity, a total of 31 million FMG tokens will be distributed via the “liquidity mining” program over two years. Initially, we will launch an FMG-ETH Uniswap Pool and provide liquidity mining incentives to it. The epoch size for FMG distribution will be one week for all the pools. For users to be eligible to earn FMG tokens, they must stake liquidity until the end of the epoch.
Initial farming parameters will be shared near the TGE. The community FMG distribution parameters will be later adjusted based on community vote once the protocol moves towards being a DAO.
Thank you for your patience. We’ve been working closely with Poolz and Bounce teams, and have obtained the full lists of all IDO participants who completed the subscription. We will be airdropping the $FMG tokens to those addresses in a short while.
We have also launched our official Uniswap pool (smoothly this time), and here’re the details:
FMG Official Token Address
0x2991341D28Eaea277785D20e1d878D478c7bA4C7
Trade on Uniswap
https://info.uniswap.org/pair/0x8815ee66cbb8fd7adaf199cc5564eb2ac52a9c00
Dextools
https://www.dextools.io/app/uniswap/pair-explorer/0x8815ee66cbb8fd7adaf199cc5564eb2ac52a9c00
Would you like to earn TOKEN right now! ☞ CLICK HERE
How and Where to Buy FM Gallery (FMG)?
FMG has been listed on a number of crypto exchanges, unlike other main cryptocurrencies, it cannot be directly purchased with fiats money. However, You can still easily buy this coin by first buying Bitcoin, ETH, USDT from any large exchanges and then transfer to the exchange that offers to trade this coin, in this guide article we will walk you through in detail the steps to buy FMG
You will have to first buy one of the major cryptocurrencies, usually either Bitcoin (BTC), Ethereum (ETH), Tether (USDT)…
We will use Binance Exchange here as it is one of the largest crypto exchanges that accept fiat deposits.
Binance is a popular cryptocurrency exchange which was started in China but then moved their headquarters to the crypto-friendly Island of Malta in the EU. Binance is popular for its crypto to crypto exchange services. Binance exploded onto the scene in the mania of 2017 and has since gone on to become the top crypto exchange in the world.
Once you finished the KYC process. You will be asked to add a payment method. Here you can either choose to provide a credit/debit card or use a bank transfer, and buy one of the major cryptocurrencies, usually either Bitcoin (BTC), Ethereum (ETH), Tether (USDT)
Step by Step Guide : What is Binance | How to Create an account on Binance (Updated 2021)
Next step - Transfer your cryptos to an Altcoin Exchange
Since FMG is an altcoin we need to transfer our coins to an exchange that FMG can be traded. Below is a list of exchanges that offers to trade FMG in various market pairs, head to their websites and register for an account.
Once finished you will then need to make a BTC/ETH/USDT deposit to the exchange from Binance depending on the available market pairs. After the deposit is confirmed you may then purchase FMG from the exchange: Uniswap (V2)
Apart from the exchange(s) above, there are a few popular crypto exchanges where they have decent daily trading volumes and a huge user base. This will ensure you will be able to sell your coins at any time and the fees will usually be lower. It is suggested that you also register on these exchanges since once FMG gets listed there it will attract a large amount of trading volumes from the users there, that means you will be having some great trading opportunities!
Top exchanges for token-coin trading. Follow instructions and make unlimited money
☞ https://www.binance.com
☞ https://www.bittrex.com
☞ https://www.poloniex.com
☞ https://www.bitfinex.com
☞ https://www.huobi.com
☞ https://www.mxc.ai
☞ https://www.probit.com
☞ https://www.gate.io
☞ https://www.coinbase.com
Find more information FMG
☞ Website ☞ Explorer ☞ Social Channel ☞ Social Channel 2 ☞ Message Board ☞ Message Board 2 ☞ Coinmarketcap
🔺DISCLAIMER: Trading Cryptocurrency is VERY risky. Make sure that you understand these risks if you are a beginner. The Information in the post is my OPINION and not financial advice. You are responsible for what you do with your funds
Learn about Cryptocurrency in this article ☞ What You Should Know Before Investing in Cryptocurrency - For Beginner
Thank for visiting and reading this article! I’m highly appreciate your actions! Please share if you liked it!
#blockchain #bitcoin #fm gallery #fmg
1659601560
Words Counted: A Ruby Natural Language Processor.
WordsCounted
We are all in the gutter, but some of us are looking at the stars.
-- Oscar Wilde
WordsCounted is a Ruby NLP (natural language processor). WordsCounted lets you implement powerful tokensation strategies with a very flexible tokeniser class.
Are you using WordsCounted to do something interesting? Please tell me about it.
Demo
Visit this website for one example of what you can do with WordsCounted.
Features
- Out of the box, get the following data from any string or readable file, or URL:
- Token count and unique token count
- Token densities, frequencies, and lengths
- Char count and average chars per token
- The longest tokens and their lengths
- The most frequent tokens and their frequencies.
- A flexible way to exclude tokens from the tokeniser. You can pass a string, regexp, symbol, lambda, or an array of any combination of those types for powerful tokenisation strategies.
- Pass your own regexp rules to the tokeniser if you prefer. The default regexp filters special characters but keeps hyphens and apostrophes. It also plays nicely with diacritics (UTF and unicode characters): Bayrūt is treated as
["Bayrūt"]and not["Bayr", "ū", "t"], for example. - Opens and reads files. Pass in a file path or a url instead of a string.
Installation
Add this line to your application's Gemfile:
gem 'words_counted'
And then execute:
$ bundle
Or install it yourself as:
$ gem install words_counted
Usage
Pass in a string or a file path, and an optional filter and/or regexp.
counter = WordsCounted.count(
"We are all in the gutter, but some of us are looking at the stars."
)
# Using a file
counter = WordsCounted.from_file("path/or/url/to/my/file.txt")
.count and .from_file are convenience methods that take an input, tokenise it, and return an instance of WordsCounted::Counter initialized with the tokens. The WordsCounted::Tokeniser and WordsCounted::Counter classes can be used alone, however.
API
WordsCounted
WordsCounted.count(input, options = {})
Tokenises input and initializes a WordsCounted::Counter object with the resulting tokens.
counter = WordsCounted.count("Hello Beirut!")
Accepts two options: exclude and regexp. See Excluding tokens from the analyser and Passing in a custom regexp respectively.
WordsCounted.from_file(path, options = {})
Reads and tokenises a file, and initializes a WordsCounted::Counter object with the resulting tokens.
counter = WordsCounted.from_file("hello_beirut.txt")
Accepts the same options as .count.
Tokeniser
The tokeniser allows you to tokenise text in a variety of ways. You can pass in your own rules for tokenisation, and apply a powerful filter with any combination of rules as long as they can boil down into a lambda.
Out of the box the tokeniser includes only alpha chars. Hyphenated tokens and tokens with apostrophes are considered a single token.
#tokenise([pattern: TOKEN_REGEXP, exclude: nil])
tokeniser = WordsCounted::Tokeniser.new("Hello Beirut!").tokenise
# With `exclude`
tokeniser = WordsCounted::Tokeniser.new("Hello Beirut!").tokenise(exclude: "hello")
# With `pattern`
tokeniser = WordsCounted::Tokeniser.new("I <3 Beirut!").tokenise(pattern: /[a-z]/i)
See Excluding tokens from the analyser and Passing in a custom regexp for more information.
Counter
The WordsCounted::Counter class allows you to collect various statistics from an array of tokens.
#token_count
Returns the token count of a given string.
counter.token_count #=> 15
#token_frequency
Returns a sorted (unstable) two-dimensional array where each element is a token and its frequency. The array is sorted by frequency in descending order.
counter.token_frequency
[
["the", 2],
["are", 2],
["we", 1],
# ...
["all", 1]
]
#most_frequent_tokens
Returns a hash where each key-value pair is a token and its frequency.
counter.most_frequent_tokens
{ "are" => 2, "the" => 2 }
#token_lengths
Returns a sorted (unstable) two-dimentional array where each element contains a token and its length. The array is sorted by length in descending order.
counter.token_lengths
[
["looking", 7],
["gutter", 6],
["stars", 5],
# ...
["in", 2]
]
#longest_tokens
Returns a hash where each key-value pair is a token and its length.
counter.longest_tokens
{ "looking" => 7 }
#token_density([ precision: 2 ])
Returns a sorted (unstable) two-dimentional array where each element contains a token and its density as a float, rounded to a precision of two. The array is sorted by density in descending order. It accepts a precision argument, which must be a float.
counter.token_density
[
["are", 0.13],
["the", 0.13],
["but", 0.07 ],
# ...
["we", 0.07 ]
]
#char_count
Returns the char count of tokens.
counter.char_count #=> 76
#average_chars_per_token([ precision: 2 ])
Returns the average char count per token rounded to two decimal places. Accepts a precision argument which defaults to two. Precision must be a float.
counter.average_chars_per_token #=> 4
#uniq_token_count
Returns the number of unique tokens.
counter.uniq_token_count #=> 13
Excluding tokens from the tokeniser
You can exclude anything you want from the input by passing the exclude option. The exclude option accepts a variety of filters and is extremely flexible.
- A space-delimited string. The filter will normalise the string.
- A regular expression.
- A lambda.
- A symbol that names a predicate method. For example
:odd?. - An array of any combination of the above.
tokeniser =
WordsCounted::Tokeniser.new(
"Magnificent! That was magnificent, Trevor."
)
# Using a string
tokeniser.tokenise(exclude: "was magnificent")
# => ["that", "trevor"]
# Using a regular expression
tokeniser.tokenise(exclude: /trevor/)
# => ["magnificent", "that", "was", "magnificent"]
# Using a lambda
tokeniser.tokenise(exclude: ->(t) { t.length < 4 })
# => ["magnificent", "that", "magnificent", "trevor"]
# Using symbol
tokeniser = WordsCounted::Tokeniser.new("Hello! محمد")
tokeniser.tokenise(exclude: :ascii_only?)
# => ["محمد"]
# Using an array
tokeniser = WordsCounted::Tokeniser.new(
"Hello! اسماءنا هي محمد، كارولينا، سامي، وداني"
)
tokeniser.tokenise(
exclude: [:ascii_only?, /محمد/, ->(t) { t.length > 6}, "و"]
)
# => ["هي", "سامي", "وداني"]
Passing in a custom regexp
The default regexp accounts for letters, hyphenated tokens, and apostrophes. This means twenty-one is treated as one token. So is Mohamad's.
/[\p{Alpha}\-']+/
You can pass your own criteria as a Ruby regular expression to split your string as desired.
For example, if you wanted to include numbers, you can override the regular expression:
counter = WordsCounted.count("Numbers 1, 2, and 3", pattern: /[\p{Alnum}\-']+/)
counter.tokens
#=> ["numbers", "1", "2", "and", "3"]
Opening and reading files
Use the from_file method to open files. from_file accepts the same options as .count. The file path can be a URL.
counter = WordsCounted.from_file("url/or/path/to/file.text")
Gotchas
A hyphen used in leu of an em or en dash will form part of the token. This affects the tokeniser algorithm.
counter = WordsCounted.count("How do you do?-you are well, I see.")
counter.token_frequency
[
["do", 2],
["how", 1],
["you", 1],
["-you", 1], # WTF, mate!
["are", 1],
# ...
]
In this example -you and you are separate tokens. Also, the tokeniser does not include numbers by default. Remember that you can pass your own regular expression if the default behaviour does not fit your needs.
A note on case sensitivity
The program will normalise (downcase) all incoming strings for consistency and filters.
Roadmap
Ability to open URLs
def self.from_url
# open url and send string here after removing html
end
Contributors
See contributors.
Contributing
- Fork it
- Create your feature branch (
git checkout -b my-new-feature) - Commit your changes (
git commit -am 'Add some feature') - Push to the branch (
git push origin my-new-feature) - Create new Pull Request
Author: abitdodgy
Source code: https://github.com/abitdodgy/words_counted
License: MIT license
#ruby #ruby-on-rails
1658068560
WordsCounted: A Ruby Natural Language Processor
WordsCounted
We are all in the gutter, but some of us are looking at the stars.
-- Oscar Wilde
WordsCounted is a Ruby NLP (natural language processor). WordsCounted lets you implement powerful tokensation strategies with a very flexible tokeniser class.
Features
- Out of the box, get the following data from any string or readable file, or URL:
- Token count and unique token count
- Token densities, frequencies, and lengths
- Char count and average chars per token
- The longest tokens and their lengths
- The most frequent tokens and their frequencies.
- A flexible way to exclude tokens from the tokeniser. You can pass a string, regexp, symbol, lambda, or an array of any combination of those types for powerful tokenisation strategies.
- Pass your own regexp rules to the tokeniser if you prefer. The default regexp filters special characters but keeps hyphens and apostrophes. It also plays nicely with diacritics (UTF and unicode characters): Bayrūt is treated as
["Bayrūt"]and not["Bayr", "ū", "t"], for example. - Opens and reads files. Pass in a file path or a url instead of a string.
Installation
Add this line to your application's Gemfile:
gem 'words_counted'
And then execute:
$ bundle
Or install it yourself as:
$ gem install words_counted
Usage
Pass in a string or a file path, and an optional filter and/or regexp.
counter = WordsCounted.count(
"We are all in the gutter, but some of us are looking at the stars."
)
# Using a file
counter = WordsCounted.from_file("path/or/url/to/my/file.txt")
.count and .from_file are convenience methods that take an input, tokenise it, and return an instance of WordsCounted::Counter initialized with the tokens. The WordsCounted::Tokeniser and WordsCounted::Counter classes can be used alone, however.
API
WordsCounted
WordsCounted.count(input, options = {})
Tokenises input and initializes a WordsCounted::Counter object with the resulting tokens.
counter = WordsCounted.count("Hello Beirut!")
Accepts two options: exclude and regexp. See Excluding tokens from the analyser and Passing in a custom regexp respectively.
WordsCounted.from_file(path, options = {})
Reads and tokenises a file, and initializes a WordsCounted::Counter object with the resulting tokens.
counter = WordsCounted.from_file("hello_beirut.txt")
Accepts the same options as .count.
Tokeniser
The tokeniser allows you to tokenise text in a variety of ways. You can pass in your own rules for tokenisation, and apply a powerful filter with any combination of rules as long as they can boil down into a lambda.
Out of the box the tokeniser includes only alpha chars. Hyphenated tokens and tokens with apostrophes are considered a single token.
#tokenise([pattern: TOKEN_REGEXP, exclude: nil])
tokeniser = WordsCounted::Tokeniser.new("Hello Beirut!").tokenise
# With `exclude`
tokeniser = WordsCounted::Tokeniser.new("Hello Beirut!").tokenise(exclude: "hello")
# With `pattern`
tokeniser = WordsCounted::Tokeniser.new("I <3 Beirut!").tokenise(pattern: /[a-z]/i)
See Excluding tokens from the analyser and Passing in a custom regexp for more information.
Counter
The WordsCounted::Counter class allows you to collect various statistics from an array of tokens.
#token_count
Returns the token count of a given string.
counter.token_count #=> 15
#token_frequency
Returns a sorted (unstable) two-dimensional array where each element is a token and its frequency. The array is sorted by frequency in descending order.
counter.token_frequency
[
["the", 2],
["are", 2],
["we", 1],
# ...
["all", 1]
]
#most_frequent_tokens
Returns a hash where each key-value pair is a token and its frequency.
counter.most_frequent_tokens
{ "are" => 2, "the" => 2 }
#token_lengths
Returns a sorted (unstable) two-dimentional array where each element contains a token and its length. The array is sorted by length in descending order.
counter.token_lengths
[
["looking", 7],
["gutter", 6],
["stars", 5],
# ...
["in", 2]
]
#longest_tokens
Returns a hash where each key-value pair is a token and its length.
counter.longest_tokens
{ "looking" => 7 }
#token_density([ precision: 2 ])
Returns a sorted (unstable) two-dimentional array where each element contains a token and its density as a float, rounded to a precision of two. The array is sorted by density in descending order. It accepts a precision argument, which must be a float.
counter.token_density
[
["are", 0.13],
["the", 0.13],
["but", 0.07 ],
# ...
["we", 0.07 ]
]
#char_count
Returns the char count of tokens.
counter.char_count #=> 76
#average_chars_per_token([ precision: 2 ])
Returns the average char count per token rounded to two decimal places. Accepts a precision argument which defaults to two. Precision must be a float.
counter.average_chars_per_token #=> 4
#uniq_token_count
Returns the number of unique tokens.
counter.uniq_token_count #=> 13
Excluding tokens from the tokeniser
You can exclude anything you want from the input by passing the exclude option. The exclude option accepts a variety of filters and is extremely flexible.
- A space-delimited string. The filter will normalise the string.
- A regular expression.
- A lambda.
- A symbol that names a predicate method. For example
:odd?. - An array of any combination of the above.
tokeniser =
WordsCounted::Tokeniser.new(
"Magnificent! That was magnificent, Trevor."
)
# Using a string
tokeniser.tokenise(exclude: "was magnificent")
# => ["that", "trevor"]
# Using a regular expression
tokeniser.tokenise(exclude: /trevor/)
# => ["magnificent", "that", "was", "magnificent"]
# Using a lambda
tokeniser.tokenise(exclude: ->(t) { t.length < 4 })
# => ["magnificent", "that", "magnificent", "trevor"]
# Using symbol
tokeniser = WordsCounted::Tokeniser.new("Hello! محمد")
tokeniser.tokenise(exclude: :ascii_only?)
# => ["محمد"]
# Using an array
tokeniser = WordsCounted::Tokeniser.new(
"Hello! اسماءنا هي محمد، كارولينا، سامي، وداني"
)
tokeniser.tokenise(
exclude: [:ascii_only?, /محمد/, ->(t) { t.length > 6}, "و"]
)
# => ["هي", "سامي", "وداني"]
Passing in a custom regexp
The default regexp accounts for letters, hyphenated tokens, and apostrophes. This means twenty-one is treated as one token. So is Mohamad's.
/[\p{Alpha}\-']+/
You can pass your own criteria as a Ruby regular expression to split your string as desired.
For example, if you wanted to include numbers, you can override the regular expression:
counter = WordsCounted.count("Numbers 1, 2, and 3", pattern: /[\p{Alnum}\-']+/)
counter.tokens
#=> ["numbers", "1", "2", "and", "3"]
Opening and reading files
Use the from_file method to open files. from_file accepts the same options as .count. The file path can be a URL.
counter = WordsCounted.from_file("url/or/path/to/file.text")
Gotchas
A hyphen used in leu of an em or en dash will form part of the token. This affects the tokeniser algorithm.
counter = WordsCounted.count("How do you do?-you are well, I see.")
counter.token_frequency
[
["do", 2],
["how", 1],
["you", 1],
["-you", 1], # WTF, mate!
["are", 1],
# ...
]
In this example -you and you are separate tokens. Also, the tokeniser does not include numbers by default. Remember that you can pass your own regular expression if the default behaviour does not fit your needs.
A note on case sensitivity
The program will normalise (downcase) all incoming strings for consistency and filters.
Roadmap
Ability to open URLs
def self.from_url
# open url and send string here after removing html
end
Are you using WordsCounted to do something interesting? Please tell me about it.
Demo
Visit this website for one example of what you can do with WordsCounted.
Contributors
See contributors.
Contributing
- Fork it
- Create your feature branch (
git checkout -b my-new-feature) - Commit your changes (
git commit -am 'Add some feature') - Push to the branch (
git push origin my-new-feature) - Create new Pull Request
Author: Abitdodgy
Source Code: https://github.com/abitdodgy/words_counted
License: MIT license
1622197808
SafeMoon Clone | Create A DeFi Token Like SafeMoon | DeFi token like SafeMoon
SafeMoon is a decentralized finance (DeFi) token. This token consists of RFI tokenomics and auto-liquidity generating protocol. A DeFi token like SafeMoon has reached the mainstream standards under the Binance Smart Chain. Its success and popularity have been immense, thus, making the majority of the business firms adopt this style of cryptocurrency as an alternative.
A DeFi token like SafeMoon is almost similar to the other crypto-token, but the only difference being that it charges a 10% transaction fee from the users who sell their tokens, in which 5% of the fee is distributed to the remaining SafeMoon owners. This feature rewards the owners for holding onto their tokens.
Read More @ https://bit.ly/3oFbJoJ
#create a defi token like safemoon #defi token like safemoon #safemoon token #safemoon token clone #defi token
1621844791
SafeMoon Clone | SafeMoon Token Clone | SafeMoon Token Clone Development
The SafeMoon Token Clone Development is the new trendsetter in the digital world that brought significant changes to benefit the growth of investors’ business in a short period. The SafeMoon token clone is the most widely discussed topic among global users for its value soaring high in the marketplace. The SafeMoon token development is a combination of RFI tokenomics and the auto-liquidity generating process. The SafeMoon token is a replica of decentralized finance (DeFi) tokens that are highly scalable and implemented with tamper-proof security.
The SafeMoon tokens execute efficient functionalities like RFI Static Rewards, Automated Liquidity Provisions, and Automatic Token Burns. The SafeMoon token is considered the most advanced stable coin in the crypto market. It gained global audience attention for managing the stability of asset value without any fluctuations in the marketplace. The SafeMoon token clone is completely decentralized that eliminates the need for intermediaries and benefits the users with less transaction fee and wait time to overtake the traditional banking process.
Reasons to invest in SafeMoon Token Clone :
- The SafeMoon token clone benefits the investors with Automated Liquidity Pool as a unique feature since it adds more revenue for their business growth in less time. The traders can experience instant trade round the clock for reaping profits with less investment towards the SafeMoon token.
- It is integrated with high-end security protocols like two-factor authentication and signature process to prevent various hacks and vulnerable activities. The Smart Contract system in SafeMoon token development manages the overall operation of transactions without any delay,
- The users can obtain a reward amount based on the volume of SafeMoon tokens traded in the marketplace. The efficient trading mechanism allows the users to trade the SafeMoon tokens at the best price for farming. The user can earn higher rewards based on the staking volume of tokens by users in the trade market.
- It allows the token holders to gain complete ownership over their SafeMoon tokens after purchasing from DeFi exchanges. The SafeMoon community governs the token distribution, price fluctuations, staking, and every other token activity. The community boosts the value of SafeMoon tokens.
- The Automated Burning tokens result in the community no longer having control over the SafeMoon tokens. Instead, the community can control the burn of the tokens efficiently for promoting its value in the marketplace. The transaction of SafeMoon tokens on the blockchain platform is fast, safe, and secure.
The SafeMoon Token Clone Development is a promising future for upcoming investors and startups to increase their business revenue in less time. The SafeMoon token clone has great demand in the real world among millions of users for its value in the market. Investors can contact leading Infinite Block Tech to gain proper assistance in developing a world-class SafeMoon token clone that increases the business growth in less time.
#safemoon token #safemoon token clone #safemoon token clone development #defi token