1612443562
What is Strudel Finance (TRDL) | What is Strudel Finance token | What is TRDL token
Intro
Centralized Bitcoin bridges expose the Ethereum ecosystem to systemic risk. Projects like REN and WBTC lock BTC in insecure multisig setups and raise concerns of total value collapse.
Strudel is the first one-way, trustless bridge linking Bitcoin to Ethereum. vBTC, the resulting asset, trades off counterparty risk for market risk, bringing more diversity to the tokenized-BTC landscape. In the long term Strudel is about using the forces of crypto-economics to challenge the status quo of blockchain and free BTC from the grip of Wall Street.
Transforming BTC to vBTC
By using the Strudel Dapp, Bitcoins are burned and marked with the Strudel protocol identifier. The burn transaction is then relayed to Ethereum, issuing vBTC in the exact ratio of 1:1.
Use on your own risk. Your Bitcoins will be burned! Read more here.
$TRDL
$TRDL (/ˈstruːdəl/) is the governance and reward token for the Strudel Protocol. It is created on 2 separate occasions:
- When a user crosses BTC over the bridge, $TRDLs are minted alongside vBTC. Early users receive an exponential reward as depicted in the graph.
- $TRDL rewards are distributed per block to liquidity providers of various terra-farming pools. Of those, the vBTC-ETH pool takes a special role in maintaining the price peg.
🚀 Terra-farming starts at block #11.021.639 with 1 $TRDL per block.
🎉 Until Block #11.434.002 - December 11 there will be a 4x $TRDL bonus 🎉
_👉 Enter the Strudel here: _https://strudel.finance
vBTC Price Stability
The Strudel Protocol implements a unique pegging algorithm keeping vBTC prices synced with BTC. Those willing to be slightly leveraged-short on BTC, deposit liquidity into the vBTC/ETH pool to earn fees by facilitating market making activities.
The algorithm utilizes Balancer Smart Pools, Uniswap Price Oracles and the flash-loan capability of vBTC. Continuous trades issued by the peg algorithm stabilize the price and provide a price guarantee within limited bounds.
- When ETH and BTC market prices correlate, liquidity providers simply earn rewards.
- When ETH goes up against BTC the pegging algorithm issues trades. Those profits are used to buy back $TRDL and burn it.
- When ETH goes down against BTC the liquidity is used to support the market price through arbitrage trading. The peg can be guaranteed up to a price disparity of 300% starting from todays price. Only when this threshold is exceeded, the arbitrage stops, and the token starts to float free.
Nowadays market conditions are optimally aligned for the proposition the Strudel protocol is making. While ETH is steadily gaining market dominance, the tokenized BTC supply of projects like WETH, REN and SNX is going through the roof.
Bridging BTC into DeFi without Intermediaries
The supply of tokenized Bitcoin on Ethereum relayed through centralized, trusted bridges now tops $1.1B. While the demand for Bitcoin in DeFi is expected to grow, a better solution without counterparty risk for bridging is needed. This post will introduce the technical components of the trustless, one-way bridge operated by Strudel.
Strudel Bridge Design
When BTC is requested to be transferred using the Strudel Dapp, a transaction output with the transfer amount and the receiving Ethereum wallet address is created. This output is handed to the user’s Bitcoin wallet through the QR code. When a user signs a transfer the resulting transaction might look something like this.
Notice the OP_RETURN in the first output, OP_RETURN is a Bitcoin-script opcode used to mark a transaction output as invalid.
Next, the header of the Bitcoin block containing the transaction needs to be registered in the relayer contract on Ethereum, where its proof-of-work is verified and the canonical chain is extended. Simple payment verification (SPV), a protocol used by Bitcoin light clients and wallets, prevents invalid blocks from entering the relayer.
Once the transaction containing the burn has been buried under enough proof of work (6 blocks), an inclusion proof is relayed onto Ethereum. The proof is verified by the Strudel contract which mints vBTC in a ratio of 1:1 to the designated Ethereum address. The protocol strictly mints 1 vBTC for 1 burned BTC, not taking any fees for bridge crossings.
Future Burn Methods
OP_RETURN is a clean way to burn BTC, where outputs are removed from the memory of Bitcoin nodes and don’t pollute the state. Unfortunately, only few Bitcoin wallets support the payment protocol (BIP70), which is the only way to construct transactions with OP_RETURN outputs.
For those users that want to burn with a BIP70-incompatible wallet, a fallback will be provided soon. The Ethereum address will be injected into a P2SH (pay to script-hash) output instead of a valid script, creating a P2FSH (pay to fake script-hash) output. While P2FSH outputs can never be spent, the Strudel protocol can use the data embedded in them to mint vBTC to designated addresses.
Burns by P2FSH outputs can only be picked up by the same account as encoded in the output. The account has to be msg.sender of the relay transaction as seen by the vBTC contract.
Relayer Contract
The relayer contract is the heart of the bridge architecture. It is able to verify new block headers of the Bitcoin blockchain and construct a canonical chain. It acts as an on-chain implementation of a Bitcoin light client.
The Strudel team is very thankful to our friends at Summa for creating and operating the first version of such a contract. Our bridge wouldn’t be possible without the function and data that the contract provides. We want to support the operation of the contract through relayer rewards. Currently, 3 different functions of the contract are proxied by the vBTC contract and pay out rewards for valid header relays.
Opening Bitcoin to DeFi
Current Bitcoin holders have no way to let the capital in their wallets work for them — to hodl is the only way. By equipping BTC with the ERC20 interface and smart-contract interoperability it becomes more versatile - it becomes vBTC. The future for Bitcoin on the DeFi space is bright, as its supply can be locked into protocols for lending, options, and other DeFi use-cases, reducing its volatility and potentially increasing market value.
vBTC upgrades BTC with another tool to hold its ground in Blockchain finance — flash loans. Flash loans are instant uncollateralized loans for the duration of only one transaction. While other tokens need to be pooled, before they can be flash-loaned, every vBTC holder has the ability to borrow up to 21,000,000 vBTC at any time — this feature is also used in our pegging algorithm.
How to enter the Strudel
In order to make your exploration through the Strudel as secure as possible — read these instructions on how to enter first.
Mint vBTC
- To start of, simply connect the Dapp with your wallet. We support Meta Mask and WalletConnect.
- Put the desired amount of vBTC, that you want to mint, in the interface. Check your vBTC and the estimated $TRDL rewards you gain. The Strudel contract strictly mints BTC to vBTC at a ratio of 1:1. Tor early explorers, the $TRDL rewards are distributed at a quadratic rate — so be quick.
- Once you continue you can see an individual QR code, based on your inputs and the connected wallet. You can either scan the code or copy & paste the payment link in your Bitcoin wallet’s recipient input field.
Compatible Wallets
Only BIP70 compatible wallets can cross the bridge. The following wallets have been tested and work properly:
- Bitcoin.com ( iOS or Android)
- Mycelium ( iOS only)
To allow all wallets to enter the Strudel a P2FSH implementation is in the making.
Claim vBTC and $TRDL rewards
Once you have entered the Strudel — the contract will wait until 6 block confirmations on the Bitcoin Blockchain. The interface provides an update during this process. Once confirmed, you can claim your vBTC and $TRDL rewards.
❗️The longer you wait to claim your tokens, the more it will cost. This is due to the relayer function of the bridge. It relays every Bitcoin block header on Ethereum — the longer it has to go back in history to check the inclusion proof the more it will cost.❗️
Immediate claim: 200.000 gas
2 week claim: 2.000.000 gas
Maximum claim time: 3 weeks!
Start Farming the Flippening
You can put your vBTC and $TRDL into DeFi immediately and support the project by providing liquidity. Strudel makes use of crypto-economic incentives to strengthen the trustless bridge and the protocol. When you put liquidity pool tokens into the Strudel — 🛰 Terra-Farms — you stabilize the peg of BTC to vBTC and earn rewards by taking a short position on BTC dominance and go long on Ethereum. You can read more about that here.
- Deposit Liquidity into the major pools of vBTC-WETH or $TRDL-ETH. You will need both of the assets to stake liquidity. For vBTC-WETH you will need vBTC and wrapped Ether. For $TRDL-ETH you need $TRDL and ETH.
- Once you staked the desired amount of vBTC and WETH or $TRDL and ETH you will get LP tokens (Liquidity Provider Tokens). The LP token of vBTC-WETH is called vBTC++.
- Now that you have the respective LP token you can stake the respective LP tokens in the reward pools vBTC-WETH LP or $TRDL-ETH and farm $TRDL.
- Earn $TRDL rewards — harvest your rewards, enter more liquidity and repeat.
You have now successfully crossed the first one-way, trustless bridge and earn rewards.
Looking for more information…
☞ Website ☞ Explorer ☞ Explorer 2 ☞ Source Code ☞ Social Channel ☞ Social Channel 2 ☞ Social Channel 3 ☞ Message Board ☞ Coinmarketcap
Would you like to earn TRDL right now! ☞ CLICK HERE
*Top exchanges for token-coin trading. Follow instructions and make unlimited money *
☞ Binance ☞ Bittrex ☞ Poloniex ☞ Bitfinex ☞ Huobi
Thank you for reading!
#bitcoin #crypto #strudel finance #trdl
What is GEEK
Buddha Community
1612443562
What is Strudel Finance (TRDL) | What is Strudel Finance token | What is TRDL token
Intro
Centralized Bitcoin bridges expose the Ethereum ecosystem to systemic risk. Projects like REN and WBTC lock BTC in insecure multisig setups and raise concerns of total value collapse.
Strudel is the first one-way, trustless bridge linking Bitcoin to Ethereum. vBTC, the resulting asset, trades off counterparty risk for market risk, bringing more diversity to the tokenized-BTC landscape. In the long term Strudel is about using the forces of crypto-economics to challenge the status quo of blockchain and free BTC from the grip of Wall Street.
Transforming BTC to vBTC
By using the Strudel Dapp, Bitcoins are burned and marked with the Strudel protocol identifier. The burn transaction is then relayed to Ethereum, issuing vBTC in the exact ratio of 1:1.
Use on your own risk. Your Bitcoins will be burned! Read more here.
$TRDL
$TRDL (/ˈstruːdəl/) is the governance and reward token for the Strudel Protocol. It is created on 2 separate occasions:
- When a user crosses BTC over the bridge, $TRDLs are minted alongside vBTC. Early users receive an exponential reward as depicted in the graph.
- $TRDL rewards are distributed per block to liquidity providers of various terra-farming pools. Of those, the vBTC-ETH pool takes a special role in maintaining the price peg.
🚀 Terra-farming starts at block #11.021.639 with 1 $TRDL per block.
🎉 Until Block #11.434.002 - December 11 there will be a 4x $TRDL bonus 🎉
_👉 Enter the Strudel here: _https://strudel.finance
vBTC Price Stability
The Strudel Protocol implements a unique pegging algorithm keeping vBTC prices synced with BTC. Those willing to be slightly leveraged-short on BTC, deposit liquidity into the vBTC/ETH pool to earn fees by facilitating market making activities.
The algorithm utilizes Balancer Smart Pools, Uniswap Price Oracles and the flash-loan capability of vBTC. Continuous trades issued by the peg algorithm stabilize the price and provide a price guarantee within limited bounds.
- When ETH and BTC market prices correlate, liquidity providers simply earn rewards.
- When ETH goes up against BTC the pegging algorithm issues trades. Those profits are used to buy back $TRDL and burn it.
- When ETH goes down against BTC the liquidity is used to support the market price through arbitrage trading. The peg can be guaranteed up to a price disparity of 300% starting from todays price. Only when this threshold is exceeded, the arbitrage stops, and the token starts to float free.
Nowadays market conditions are optimally aligned for the proposition the Strudel protocol is making. While ETH is steadily gaining market dominance, the tokenized BTC supply of projects like WETH, REN and SNX is going through the roof.
Bridging BTC into DeFi without Intermediaries
The supply of tokenized Bitcoin on Ethereum relayed through centralized, trusted bridges now tops $1.1B. While the demand for Bitcoin in DeFi is expected to grow, a better solution without counterparty risk for bridging is needed. This post will introduce the technical components of the trustless, one-way bridge operated by Strudel.
Strudel Bridge Design
When BTC is requested to be transferred using the Strudel Dapp, a transaction output with the transfer amount and the receiving Ethereum wallet address is created. This output is handed to the user’s Bitcoin wallet through the QR code. When a user signs a transfer the resulting transaction might look something like this.
Notice the OP_RETURN in the first output, OP_RETURN is a Bitcoin-script opcode used to mark a transaction output as invalid.
Next, the header of the Bitcoin block containing the transaction needs to be registered in the relayer contract on Ethereum, where its proof-of-work is verified and the canonical chain is extended. Simple payment verification (SPV), a protocol used by Bitcoin light clients and wallets, prevents invalid blocks from entering the relayer.
Once the transaction containing the burn has been buried under enough proof of work (6 blocks), an inclusion proof is relayed onto Ethereum. The proof is verified by the Strudel contract which mints vBTC in a ratio of 1:1 to the designated Ethereum address. The protocol strictly mints 1 vBTC for 1 burned BTC, not taking any fees for bridge crossings.
Future Burn Methods
OP_RETURN is a clean way to burn BTC, where outputs are removed from the memory of Bitcoin nodes and don’t pollute the state. Unfortunately, only few Bitcoin wallets support the payment protocol (BIP70), which is the only way to construct transactions with OP_RETURN outputs.
For those users that want to burn with a BIP70-incompatible wallet, a fallback will be provided soon. The Ethereum address will be injected into a P2SH (pay to script-hash) output instead of a valid script, creating a P2FSH (pay to fake script-hash) output. While P2FSH outputs can never be spent, the Strudel protocol can use the data embedded in them to mint vBTC to designated addresses.
Burns by P2FSH outputs can only be picked up by the same account as encoded in the output. The account has to be msg.sender of the relay transaction as seen by the vBTC contract.
Relayer Contract
The relayer contract is the heart of the bridge architecture. It is able to verify new block headers of the Bitcoin blockchain and construct a canonical chain. It acts as an on-chain implementation of a Bitcoin light client.
The Strudel team is very thankful to our friends at Summa for creating and operating the first version of such a contract. Our bridge wouldn’t be possible without the function and data that the contract provides. We want to support the operation of the contract through relayer rewards. Currently, 3 different functions of the contract are proxied by the vBTC contract and pay out rewards for valid header relays.
Opening Bitcoin to DeFi
Current Bitcoin holders have no way to let the capital in their wallets work for them — to hodl is the only way. By equipping BTC with the ERC20 interface and smart-contract interoperability it becomes more versatile - it becomes vBTC. The future for Bitcoin on the DeFi space is bright, as its supply can be locked into protocols for lending, options, and other DeFi use-cases, reducing its volatility and potentially increasing market value.
vBTC upgrades BTC with another tool to hold its ground in Blockchain finance — flash loans. Flash loans are instant uncollateralized loans for the duration of only one transaction. While other tokens need to be pooled, before they can be flash-loaned, every vBTC holder has the ability to borrow up to 21,000,000 vBTC at any time — this feature is also used in our pegging algorithm.
How to enter the Strudel
In order to make your exploration through the Strudel as secure as possible — read these instructions on how to enter first.
Mint vBTC
- To start of, simply connect the Dapp with your wallet. We support Meta Mask and WalletConnect.
- Put the desired amount of vBTC, that you want to mint, in the interface. Check your vBTC and the estimated $TRDL rewards you gain. The Strudel contract strictly mints BTC to vBTC at a ratio of 1:1. Tor early explorers, the $TRDL rewards are distributed at a quadratic rate — so be quick.
- Once you continue you can see an individual QR code, based on your inputs and the connected wallet. You can either scan the code or copy & paste the payment link in your Bitcoin wallet’s recipient input field.
Compatible Wallets
Only BIP70 compatible wallets can cross the bridge. The following wallets have been tested and work properly:
- Bitcoin.com ( iOS or Android)
- Mycelium ( iOS only)
To allow all wallets to enter the Strudel a P2FSH implementation is in the making.
Claim vBTC and $TRDL rewards
Once you have entered the Strudel — the contract will wait until 6 block confirmations on the Bitcoin Blockchain. The interface provides an update during this process. Once confirmed, you can claim your vBTC and $TRDL rewards.
❗️The longer you wait to claim your tokens, the more it will cost. This is due to the relayer function of the bridge. It relays every Bitcoin block header on Ethereum — the longer it has to go back in history to check the inclusion proof the more it will cost.❗️
Immediate claim: 200.000 gas
2 week claim: 2.000.000 gas
Maximum claim time: 3 weeks!
Start Farming the Flippening
You can put your vBTC and $TRDL into DeFi immediately and support the project by providing liquidity. Strudel makes use of crypto-economic incentives to strengthen the trustless bridge and the protocol. When you put liquidity pool tokens into the Strudel — 🛰 Terra-Farms — you stabilize the peg of BTC to vBTC and earn rewards by taking a short position on BTC dominance and go long on Ethereum. You can read more about that here.
- Deposit Liquidity into the major pools of vBTC-WETH or $TRDL-ETH. You will need both of the assets to stake liquidity. For vBTC-WETH you will need vBTC and wrapped Ether. For $TRDL-ETH you need $TRDL and ETH.
- Once you staked the desired amount of vBTC and WETH or $TRDL and ETH you will get LP tokens (Liquidity Provider Tokens). The LP token of vBTC-WETH is called vBTC++.
- Now that you have the respective LP token you can stake the respective LP tokens in the reward pools vBTC-WETH LP or $TRDL-ETH and farm $TRDL.
- Earn $TRDL rewards — harvest your rewards, enter more liquidity and repeat.
You have now successfully crossed the first one-way, trustless bridge and earn rewards.
Looking for more information…
☞ Website ☞ Explorer ☞ Explorer 2 ☞ Source Code ☞ Social Channel ☞ Social Channel 2 ☞ Social Channel 3 ☞ Message Board ☞ Coinmarketcap
Would you like to earn TRDL right now! ☞ CLICK HERE
*Top exchanges for token-coin trading. Follow instructions and make unlimited money *
☞ Binance ☞ Bittrex ☞ Poloniex ☞ Bitfinex ☞ Huobi
Thank you for reading!
#bitcoin #crypto #strudel finance #trdl
1624219980
How to Buy NFT Art Finance Token - The EASIEST METHOD! DO NOT MISS!!! JUST IN 4 MINUTES
NFT Art Finance is currently one of the most popular cryptocurrencies right now on the market, so in today’s video, I will be showing you guys how to easily buy NFT Art Finance on your phone using the Trust Wallet application.
📺 The video in this post was made by More LimSanity
The origin of the article: https://www.youtube.com/watch?v=sKE6Pc_w1IE
🔺 DISCLAIMER: The article is for information sharing. The content of this video is solely the opinions of the speaker who is not a licensed financial advisor or registered investment advisor. Not investment advice or legal advice.
Cryptocurrency trading is VERY risky. Make sure you understand these risks and that you are responsible for what you do with your money
🔥 If you’re a beginner. I believe the article below will be useful to you ☞ What You Should Know Before Investing in Cryptocurrency - For Beginner
⭐ ⭐ ⭐The project is of interest to the community. Join to Get free ‘GEEK coin’ (GEEKCASH coin)!
☞ **-----CLICK HERE-----**⭐ ⭐ ⭐
Thanks for visiting and watching! Please don’t forget to leave a like, comment and share!
#bitcoin #blockchain #nft art finance token #token #buy nft art finance #how to buy nft art finance token - the easiest method!
1624312800
SPORE FINANCE PREDICTION - WHAT IS SPORE FINANCE & SPORE FINANCE ANALYSIS - SPORE FINANCE
SPORE FINANCE PREDICTION - WHAT IS SPORE FINANCE & SPORE FINANCE ANALYSIS - SPORE FINANCE
In this video, I talk about spore finance coin and give my spore finance prediction. I talk about the latest spore finance analysis & spore finance crypto coin that recently has been hit pretty hard in the last 24 hours. I go over what is spore finance and how many holders are on this new crypto coin spore finance.
📺 The video in this post was made by Josh’s Finance
The origin of the article: https://www.youtube.com/watch?v=qbPQvdxCtEI
🔺 DISCLAIMER: The article is for information sharing. The content of this video is solely the opinions of the speaker who is not a licensed financial advisor or registered investment advisor. Not investment advice or legal advice.
Cryptocurrency trading is VERY risky. Make sure you understand these risks and that you are responsible for what you do with your money
🔥 If you’re a beginner. I believe the article below will be useful to you ☞ What You Should Know Before Investing in Cryptocurrency - For Beginner
⭐ ⭐ ⭐The project is of interest to the community. Join to Get free ‘GEEK coin’ (GEEKCASH coin)!
☞ **-----CLICK HERE-----**⭐ ⭐ ⭐
Thanks for visiting and watching! Please don’t forget to leave a like, comment and share!
#bitcoin #blockchain #spore finance #what is spore finance #spore finance prediction - what is spore finance & spore finance analysis - spore finance #spore finance prediction
1658068560
WordsCounted: A Ruby Natural Language Processor
WordsCounted
We are all in the gutter, but some of us are looking at the stars.
-- Oscar Wilde
WordsCounted is a Ruby NLP (natural language processor). WordsCounted lets you implement powerful tokensation strategies with a very flexible tokeniser class.
Features
- Out of the box, get the following data from any string or readable file, or URL:
- Token count and unique token count
- Token densities, frequencies, and lengths
- Char count and average chars per token
- The longest tokens and their lengths
- The most frequent tokens and their frequencies.
- A flexible way to exclude tokens from the tokeniser. You can pass a string, regexp, symbol, lambda, or an array of any combination of those types for powerful tokenisation strategies.
- Pass your own regexp rules to the tokeniser if you prefer. The default regexp filters special characters but keeps hyphens and apostrophes. It also plays nicely with diacritics (UTF and unicode characters): Bayrūt is treated as
["Bayrūt"]and not["Bayr", "ū", "t"], for example. - Opens and reads files. Pass in a file path or a url instead of a string.
Installation
Add this line to your application's Gemfile:
gem 'words_counted'
And then execute:
$ bundle
Or install it yourself as:
$ gem install words_counted
Usage
Pass in a string or a file path, and an optional filter and/or regexp.
counter = WordsCounted.count(
"We are all in the gutter, but some of us are looking at the stars."
)
# Using a file
counter = WordsCounted.from_file("path/or/url/to/my/file.txt")
.count and .from_file are convenience methods that take an input, tokenise it, and return an instance of WordsCounted::Counter initialized with the tokens. The WordsCounted::Tokeniser and WordsCounted::Counter classes can be used alone, however.
API
WordsCounted
WordsCounted.count(input, options = {})
Tokenises input and initializes a WordsCounted::Counter object with the resulting tokens.
counter = WordsCounted.count("Hello Beirut!")
Accepts two options: exclude and regexp. See Excluding tokens from the analyser and Passing in a custom regexp respectively.
WordsCounted.from_file(path, options = {})
Reads and tokenises a file, and initializes a WordsCounted::Counter object with the resulting tokens.
counter = WordsCounted.from_file("hello_beirut.txt")
Accepts the same options as .count.
Tokeniser
The tokeniser allows you to tokenise text in a variety of ways. You can pass in your own rules for tokenisation, and apply a powerful filter with any combination of rules as long as they can boil down into a lambda.
Out of the box the tokeniser includes only alpha chars. Hyphenated tokens and tokens with apostrophes are considered a single token.
#tokenise([pattern: TOKEN_REGEXP, exclude: nil])
tokeniser = WordsCounted::Tokeniser.new("Hello Beirut!").tokenise
# With `exclude`
tokeniser = WordsCounted::Tokeniser.new("Hello Beirut!").tokenise(exclude: "hello")
# With `pattern`
tokeniser = WordsCounted::Tokeniser.new("I <3 Beirut!").tokenise(pattern: /[a-z]/i)
See Excluding tokens from the analyser and Passing in a custom regexp for more information.
Counter
The WordsCounted::Counter class allows you to collect various statistics from an array of tokens.
#token_count
Returns the token count of a given string.
counter.token_count #=> 15
#token_frequency
Returns a sorted (unstable) two-dimensional array where each element is a token and its frequency. The array is sorted by frequency in descending order.
counter.token_frequency
[
["the", 2],
["are", 2],
["we", 1],
# ...
["all", 1]
]
#most_frequent_tokens
Returns a hash where each key-value pair is a token and its frequency.
counter.most_frequent_tokens
{ "are" => 2, "the" => 2 }
#token_lengths
Returns a sorted (unstable) two-dimentional array where each element contains a token and its length. The array is sorted by length in descending order.
counter.token_lengths
[
["looking", 7],
["gutter", 6],
["stars", 5],
# ...
["in", 2]
]
#longest_tokens
Returns a hash where each key-value pair is a token and its length.
counter.longest_tokens
{ "looking" => 7 }
#token_density([ precision: 2 ])
Returns a sorted (unstable) two-dimentional array where each element contains a token and its density as a float, rounded to a precision of two. The array is sorted by density in descending order. It accepts a precision argument, which must be a float.
counter.token_density
[
["are", 0.13],
["the", 0.13],
["but", 0.07 ],
# ...
["we", 0.07 ]
]
#char_count
Returns the char count of tokens.
counter.char_count #=> 76
#average_chars_per_token([ precision: 2 ])
Returns the average char count per token rounded to two decimal places. Accepts a precision argument which defaults to two. Precision must be a float.
counter.average_chars_per_token #=> 4
#uniq_token_count
Returns the number of unique tokens.
counter.uniq_token_count #=> 13
Excluding tokens from the tokeniser
You can exclude anything you want from the input by passing the exclude option. The exclude option accepts a variety of filters and is extremely flexible.
- A space-delimited string. The filter will normalise the string.
- A regular expression.
- A lambda.
- A symbol that names a predicate method. For example
:odd?. - An array of any combination of the above.
tokeniser =
WordsCounted::Tokeniser.new(
"Magnificent! That was magnificent, Trevor."
)
# Using a string
tokeniser.tokenise(exclude: "was magnificent")
# => ["that", "trevor"]
# Using a regular expression
tokeniser.tokenise(exclude: /trevor/)
# => ["magnificent", "that", "was", "magnificent"]
# Using a lambda
tokeniser.tokenise(exclude: ->(t) { t.length < 4 })
# => ["magnificent", "that", "magnificent", "trevor"]
# Using symbol
tokeniser = WordsCounted::Tokeniser.new("Hello! محمد")
tokeniser.tokenise(exclude: :ascii_only?)
# => ["محمد"]
# Using an array
tokeniser = WordsCounted::Tokeniser.new(
"Hello! اسماءنا هي محمد، كارولينا، سامي، وداني"
)
tokeniser.tokenise(
exclude: [:ascii_only?, /محمد/, ->(t) { t.length > 6}, "و"]
)
# => ["هي", "سامي", "وداني"]
Passing in a custom regexp
The default regexp accounts for letters, hyphenated tokens, and apostrophes. This means twenty-one is treated as one token. So is Mohamad's.
/[\p{Alpha}\-']+/
You can pass your own criteria as a Ruby regular expression to split your string as desired.
For example, if you wanted to include numbers, you can override the regular expression:
counter = WordsCounted.count("Numbers 1, 2, and 3", pattern: /[\p{Alnum}\-']+/)
counter.tokens
#=> ["numbers", "1", "2", "and", "3"]
Opening and reading files
Use the from_file method to open files. from_file accepts the same options as .count. The file path can be a URL.
counter = WordsCounted.from_file("url/or/path/to/file.text")
Gotchas
A hyphen used in leu of an em or en dash will form part of the token. This affects the tokeniser algorithm.
counter = WordsCounted.count("How do you do?-you are well, I see.")
counter.token_frequency
[
["do", 2],
["how", 1],
["you", 1],
["-you", 1], # WTF, mate!
["are", 1],
# ...
]
In this example -you and you are separate tokens. Also, the tokeniser does not include numbers by default. Remember that you can pass your own regular expression if the default behaviour does not fit your needs.
A note on case sensitivity
The program will normalise (downcase) all incoming strings for consistency and filters.
Roadmap
Ability to open URLs
def self.from_url
# open url and send string here after removing html
end
Are you using WordsCounted to do something interesting? Please tell me about it.
Demo
Visit this website for one example of what you can do with WordsCounted.
Contributors
See contributors.
Contributing
- Fork it
- Create your feature branch (
git checkout -b my-new-feature) - Commit your changes (
git commit -am 'Add some feature') - Push to the branch (
git push origin my-new-feature) - Create new Pull Request
Author: Abitdodgy
Source Code: https://github.com/abitdodgy/words_counted
License: MIT license
1659601560
Words Counted: A Ruby Natural Language Processor.
WordsCounted
We are all in the gutter, but some of us are looking at the stars.
-- Oscar Wilde
WordsCounted is a Ruby NLP (natural language processor). WordsCounted lets you implement powerful tokensation strategies with a very flexible tokeniser class.
Are you using WordsCounted to do something interesting? Please tell me about it.
Demo
Visit this website for one example of what you can do with WordsCounted.
Features
- Out of the box, get the following data from any string or readable file, or URL:
- Token count and unique token count
- Token densities, frequencies, and lengths
- Char count and average chars per token
- The longest tokens and their lengths
- The most frequent tokens and their frequencies.
- A flexible way to exclude tokens from the tokeniser. You can pass a string, regexp, symbol, lambda, or an array of any combination of those types for powerful tokenisation strategies.
- Pass your own regexp rules to the tokeniser if you prefer. The default regexp filters special characters but keeps hyphens and apostrophes. It also plays nicely with diacritics (UTF and unicode characters): Bayrūt is treated as
["Bayrūt"]and not["Bayr", "ū", "t"], for example. - Opens and reads files. Pass in a file path or a url instead of a string.
Installation
Add this line to your application's Gemfile:
gem 'words_counted'
And then execute:
$ bundle
Or install it yourself as:
$ gem install words_counted
Usage
Pass in a string or a file path, and an optional filter and/or regexp.
counter = WordsCounted.count(
"We are all in the gutter, but some of us are looking at the stars."
)
# Using a file
counter = WordsCounted.from_file("path/or/url/to/my/file.txt")
.count and .from_file are convenience methods that take an input, tokenise it, and return an instance of WordsCounted::Counter initialized with the tokens. The WordsCounted::Tokeniser and WordsCounted::Counter classes can be used alone, however.
API
WordsCounted
WordsCounted.count(input, options = {})
Tokenises input and initializes a WordsCounted::Counter object with the resulting tokens.
counter = WordsCounted.count("Hello Beirut!")
Accepts two options: exclude and regexp. See Excluding tokens from the analyser and Passing in a custom regexp respectively.
WordsCounted.from_file(path, options = {})
Reads and tokenises a file, and initializes a WordsCounted::Counter object with the resulting tokens.
counter = WordsCounted.from_file("hello_beirut.txt")
Accepts the same options as .count.
Tokeniser
The tokeniser allows you to tokenise text in a variety of ways. You can pass in your own rules for tokenisation, and apply a powerful filter with any combination of rules as long as they can boil down into a lambda.
Out of the box the tokeniser includes only alpha chars. Hyphenated tokens and tokens with apostrophes are considered a single token.
#tokenise([pattern: TOKEN_REGEXP, exclude: nil])
tokeniser = WordsCounted::Tokeniser.new("Hello Beirut!").tokenise
# With `exclude`
tokeniser = WordsCounted::Tokeniser.new("Hello Beirut!").tokenise(exclude: "hello")
# With `pattern`
tokeniser = WordsCounted::Tokeniser.new("I <3 Beirut!").tokenise(pattern: /[a-z]/i)
See Excluding tokens from the analyser and Passing in a custom regexp for more information.
Counter
The WordsCounted::Counter class allows you to collect various statistics from an array of tokens.
#token_count
Returns the token count of a given string.
counter.token_count #=> 15
#token_frequency
Returns a sorted (unstable) two-dimensional array where each element is a token and its frequency. The array is sorted by frequency in descending order.
counter.token_frequency
[
["the", 2],
["are", 2],
["we", 1],
# ...
["all", 1]
]
#most_frequent_tokens
Returns a hash where each key-value pair is a token and its frequency.
counter.most_frequent_tokens
{ "are" => 2, "the" => 2 }
#token_lengths
Returns a sorted (unstable) two-dimentional array where each element contains a token and its length. The array is sorted by length in descending order.
counter.token_lengths
[
["looking", 7],
["gutter", 6],
["stars", 5],
# ...
["in", 2]
]
#longest_tokens
Returns a hash where each key-value pair is a token and its length.
counter.longest_tokens
{ "looking" => 7 }
#token_density([ precision: 2 ])
Returns a sorted (unstable) two-dimentional array where each element contains a token and its density as a float, rounded to a precision of two. The array is sorted by density in descending order. It accepts a precision argument, which must be a float.
counter.token_density
[
["are", 0.13],
["the", 0.13],
["but", 0.07 ],
# ...
["we", 0.07 ]
]
#char_count
Returns the char count of tokens.
counter.char_count #=> 76
#average_chars_per_token([ precision: 2 ])
Returns the average char count per token rounded to two decimal places. Accepts a precision argument which defaults to two. Precision must be a float.
counter.average_chars_per_token #=> 4
#uniq_token_count
Returns the number of unique tokens.
counter.uniq_token_count #=> 13
Excluding tokens from the tokeniser
You can exclude anything you want from the input by passing the exclude option. The exclude option accepts a variety of filters and is extremely flexible.
- A space-delimited string. The filter will normalise the string.
- A regular expression.
- A lambda.
- A symbol that names a predicate method. For example
:odd?. - An array of any combination of the above.
tokeniser =
WordsCounted::Tokeniser.new(
"Magnificent! That was magnificent, Trevor."
)
# Using a string
tokeniser.tokenise(exclude: "was magnificent")
# => ["that", "trevor"]
# Using a regular expression
tokeniser.tokenise(exclude: /trevor/)
# => ["magnificent", "that", "was", "magnificent"]
# Using a lambda
tokeniser.tokenise(exclude: ->(t) { t.length < 4 })
# => ["magnificent", "that", "magnificent", "trevor"]
# Using symbol
tokeniser = WordsCounted::Tokeniser.new("Hello! محمد")
tokeniser.tokenise(exclude: :ascii_only?)
# => ["محمد"]
# Using an array
tokeniser = WordsCounted::Tokeniser.new(
"Hello! اسماءنا هي محمد، كارولينا، سامي، وداني"
)
tokeniser.tokenise(
exclude: [:ascii_only?, /محمد/, ->(t) { t.length > 6}, "و"]
)
# => ["هي", "سامي", "وداني"]
Passing in a custom regexp
The default regexp accounts for letters, hyphenated tokens, and apostrophes. This means twenty-one is treated as one token. So is Mohamad's.
/[\p{Alpha}\-']+/
You can pass your own criteria as a Ruby regular expression to split your string as desired.
For example, if you wanted to include numbers, you can override the regular expression:
counter = WordsCounted.count("Numbers 1, 2, and 3", pattern: /[\p{Alnum}\-']+/)
counter.tokens
#=> ["numbers", "1", "2", "and", "3"]
Opening and reading files
Use the from_file method to open files. from_file accepts the same options as .count. The file path can be a URL.
counter = WordsCounted.from_file("url/or/path/to/file.text")
Gotchas
A hyphen used in leu of an em or en dash will form part of the token. This affects the tokeniser algorithm.
counter = WordsCounted.count("How do you do?-you are well, I see.")
counter.token_frequency
[
["do", 2],
["how", 1],
["you", 1],
["-you", 1], # WTF, mate!
["are", 1],
# ...
]
In this example -you and you are separate tokens. Also, the tokeniser does not include numbers by default. Remember that you can pass your own regular expression if the default behaviour does not fit your needs.
A note on case sensitivity
The program will normalise (downcase) all incoming strings for consistency and filters.
Roadmap
Ability to open URLs
def self.from_url
# open url and send string here after removing html
end
Contributors
See contributors.
Contributing
- Fork it
- Create your feature branch (
git checkout -b my-new-feature) - Commit your changes (
git commit -am 'Add some feature') - Push to the branch (
git push origin my-new-feature) - Create new Pull Request
Author: abitdodgy
Source code: https://github.com/abitdodgy/words_counted
License: MIT license
#ruby #ruby-on-rails