What is fyeth.finance (YETH) | What is YETH token

About

FYETH FINANCE is a wholly owned subsidiary of the global fintech group Investment Evolution, one of the licensed non-bank financial institutions in the United States - and operates in European countries. Platform that will allow users to borrow money in US dollars based on crypto assets. The three cryptocurrencies as collateral are BTC, ETH and yETH. The platform will operate 24/7 worldwide, Loans are available as soon as collateral is offered loan transactions and loan repayment will be paid through Blockchain. We that allows user to define their loan terms including an (LTV) ratio of 30 to 90%.Their balance sheet we “stronger than ever” and that its margins have expanded, hence it will increase deposit rates than traditional banks. We want to bring our traditional customers into the digital currency space, while targeting crypto holders in more than 200 countries and territories, who can borrow in difference multiple fiat currencies based on their digital assets. Now, with our loan products, Individual and institutional investors can transact like the big boys without selling the cryptocurrencies they have hoarded or leaving their wallets. They see us as a lever to maximize profits in cryptocurrency market transactions. We will become a global brand in digital currency loans and loans.

Why Choose yETH?

yETH vault are powerful financial primitives. yETH are tokens that constantly increase in value from earning interest, while tokens that represent short are leveraged positions. These yETH Tokens composed into novel financial products, used as collateral for loans, or listed on any exchanges to instantly enable margin lending and trading.

yETH vault is powerful financial primitives. yETH is tokens that constantly increase in value from earning interest, while tokens that represent short are leveraged positions. These yETH Tokens composed of novel financial products, used as collateral for loans, or listed on any exchanges to instantly enable margin lending and trading.

Image for post

Quick primer on how the vaults work, as there seems to be quite a bit of misinformation currently going around.

Where does the yield come from;

  1. Lending, assets are lent out via lenders such as Aave, Compound, and dYdX
  2. Trading, assets provided to Uniswap, Balancer, and Curve earn trading fees.
  3. Liquidity incentives, protocols such as Compound, Balancer, and Curve provide liquidity incentives, they provide additional incentives for liquidity

What risks are associated with this;

  1. Lending, since it is a lender, assets might not be available all the time, but that’s why lenders use utilization ratio. How this works, the closer the pool is to empty (maximum amount borrowed out) the higher the interest paid for it. This allows new assets to be provided (or borrowed assets to be returned). But the window does exist where provided assets exceed available borrowed assets.
  2. Smart contract, as always there can be bugs, exploits, and other flaws.
  3. Lack of trading activity, which means reduced fees
  4. Liquidity incentive price fluctuations causing unstable APY

yETH vault explained;

  1. yETH vault provides ETH to Maker to mint DAI.
  2. DAI is provided to yDAI (the yearn.finance DAI vault).

yDAI vault explained;

  1. DAI is provided as LP for curve.fi/y
  2. curve.fi/y LP tokens are locked in the gauge to receive CRV

We have seen a lot of misinformation around the yETH vault and its minted DAI vs available DAI. This is no different than a lender/borrower utilization ratio.

If you supply 100 tokens to a lender, and someone borrows 50 tokens. You can not withdraw your 100 tokens, but you can withdraw 50 tokens. When you withdraw 50 tokens then the borrower pays an additional premium, so other lenders are incentivized to add tokens or the borrower to repay their debt.

This is the base premise for all vaults, and yETH is no exception. There is however one difference, since yETH uses yDAI, the “lender” is in fact curve.fi.

As DAI in the pool becomes low, arbitragers sell DAI for (USDC, USDT, or TUSD). This adds DAI to the pool. So as DAI is removed, this makes it more valuable to trade in DAI. This is the same mechanism as explained for lenders and borrowers utilization ratios.

The only core difference here, is that in a normal vault, there is no debt. With yaLINK and yETH there is debt, this does add additional risk, since you need to have enough available funds to cover debt. This is why we maintain a ~200% ratio, so there is a ~50% buffer in case of lender/liquidity shortages.

Most systems have a maximum borrow buffer. This means a certain amount of liquidity minimum must be available in the system. The general rule is around ~25% (so borrowers can’t borrow more than this). For this reason the yETH vault was capped at ~60m, as this buffer is around ~$16m, which even if y pool halved in capacity, would still be available.

FYETH FINANCE FAQ

Image for post

Recently, we have been asked some of very popular questions for a multiple times. We would love to make this article, it will be well-explained a number of questions that most people ask.

How to buy the yETH?

Yes, please wait for the public sale rounds.

What is the total supply of the Fyeth finance ?

It has only 9.000.000 yETH

What is the initial circulating supply?

It has only 3.600.000 yETH.

How long is the vesting for the rest allocation?

It takes up to 6 years, the core team is only allocated for 900.000 yETH

When will the presale and public sale go live?

The presale round has ended. We are preparing for the public sale.

What are the allocations for the presale and public sale rounds?

For the presale: 540.000 yETH, for the public sale: 360.000 yETH

How does Fyeeth Finance the public sale rounds?

Round 1: 50,000 — price $3.00

Round 2: 100,000 — price $3.46

Round 3: 200.00 — price $3.96

The minimum is 100$, maximum is 5,000$.

What exchanges our yETH will be listed?

We will not disclose the names all here, but we will be listed in at least 3 Dex, and 3 centralized exchanges in Quarter 2, 2021

When can we sell our eYTH?

All YETH tokens are distributed during the pre-sale will be locked in a trading contract for a period of more than 12 months and during the public sale is 6 months.

Would you like to earn many tokens and cryptocurrencies right now! ☞ CLICK HERE

Looking for more information…

☞ Website
☞ Explorer
☞ Source Code
☞ Social Channel
Message Board
☞ Documentation
☞ Coinmarketcap

Thank for visiting and reading this article! I’m highly appreciate your actions! Please share if you liked it!

#blockchain #bitcoin #crypto #fyeth.finance #yeth

What is GEEK

Buddha Community

What is fyeth.finance (YETH) | What is YETH token

What is fyeth.finance (YETH) | What is YETH token

About

FYETH FINANCE is a wholly owned subsidiary of the global fintech group Investment Evolution, one of the licensed non-bank financial institutions in the United States - and operates in European countries. Platform that will allow users to borrow money in US dollars based on crypto assets. The three cryptocurrencies as collateral are BTC, ETH and yETH. The platform will operate 24/7 worldwide, Loans are available as soon as collateral is offered loan transactions and loan repayment will be paid through Blockchain. We that allows user to define their loan terms including an (LTV) ratio of 30 to 90%.Their balance sheet we “stronger than ever” and that its margins have expanded, hence it will increase deposit rates than traditional banks. We want to bring our traditional customers into the digital currency space, while targeting crypto holders in more than 200 countries and territories, who can borrow in difference multiple fiat currencies based on their digital assets. Now, with our loan products, Individual and institutional investors can transact like the big boys without selling the cryptocurrencies they have hoarded or leaving their wallets. They see us as a lever to maximize profits in cryptocurrency market transactions. We will become a global brand in digital currency loans and loans.

Why Choose yETH?

yETH vault are powerful financial primitives. yETH are tokens that constantly increase in value from earning interest, while tokens that represent short are leveraged positions. These yETH Tokens composed into novel financial products, used as collateral for loans, or listed on any exchanges to instantly enable margin lending and trading.

yETH vault is powerful financial primitives. yETH is tokens that constantly increase in value from earning interest, while tokens that represent short are leveraged positions. These yETH Tokens composed of novel financial products, used as collateral for loans, or listed on any exchanges to instantly enable margin lending and trading.

Image for post

Quick primer on how the vaults work, as there seems to be quite a bit of misinformation currently going around.

Where does the yield come from;

  1. Lending, assets are lent out via lenders such as Aave, Compound, and dYdX
  2. Trading, assets provided to Uniswap, Balancer, and Curve earn trading fees.
  3. Liquidity incentives, protocols such as Compound, Balancer, and Curve provide liquidity incentives, they provide additional incentives for liquidity

What risks are associated with this;

  1. Lending, since it is a lender, assets might not be available all the time, but that’s why lenders use utilization ratio. How this works, the closer the pool is to empty (maximum amount borrowed out) the higher the interest paid for it. This allows new assets to be provided (or borrowed assets to be returned). But the window does exist where provided assets exceed available borrowed assets.
  2. Smart contract, as always there can be bugs, exploits, and other flaws.
  3. Lack of trading activity, which means reduced fees
  4. Liquidity incentive price fluctuations causing unstable APY

yETH vault explained;

  1. yETH vault provides ETH to Maker to mint DAI.
  2. DAI is provided to yDAI (the yearn.finance DAI vault).

yDAI vault explained;

  1. DAI is provided as LP for curve.fi/y
  2. curve.fi/y LP tokens are locked in the gauge to receive CRV

We have seen a lot of misinformation around the yETH vault and its minted DAI vs available DAI. This is no different than a lender/borrower utilization ratio.

If you supply 100 tokens to a lender, and someone borrows 50 tokens. You can not withdraw your 100 tokens, but you can withdraw 50 tokens. When you withdraw 50 tokens then the borrower pays an additional premium, so other lenders are incentivized to add tokens or the borrower to repay their debt.

This is the base premise for all vaults, and yETH is no exception. There is however one difference, since yETH uses yDAI, the “lender” is in fact curve.fi.

As DAI in the pool becomes low, arbitragers sell DAI for (USDC, USDT, or TUSD). This adds DAI to the pool. So as DAI is removed, this makes it more valuable to trade in DAI. This is the same mechanism as explained for lenders and borrowers utilization ratios.

The only core difference here, is that in a normal vault, there is no debt. With yaLINK and yETH there is debt, this does add additional risk, since you need to have enough available funds to cover debt. This is why we maintain a ~200% ratio, so there is a ~50% buffer in case of lender/liquidity shortages.

Most systems have a maximum borrow buffer. This means a certain amount of liquidity minimum must be available in the system. The general rule is around ~25% (so borrowers can’t borrow more than this). For this reason the yETH vault was capped at ~60m, as this buffer is around ~$16m, which even if y pool halved in capacity, would still be available.

FYETH FINANCE FAQ

Image for post

Recently, we have been asked some of very popular questions for a multiple times. We would love to make this article, it will be well-explained a number of questions that most people ask.

How to buy the yETH?

Yes, please wait for the public sale rounds.

What is the total supply of the Fyeth finance ?

It has only 9.000.000 yETH

What is the initial circulating supply?

It has only 3.600.000 yETH.

How long is the vesting for the rest allocation?

It takes up to 6 years, the core team is only allocated for 900.000 yETH

When will the presale and public sale go live?

The presale round has ended. We are preparing for the public sale.

What are the allocations for the presale and public sale rounds?

For the presale: 540.000 yETH, for the public sale: 360.000 yETH

How does Fyeeth Finance the public sale rounds?

Round 1: 50,000 — price $3.00

Round 2: 100,000 — price $3.46

Round 3: 200.00 — price $3.96

The minimum is 100$, maximum is 5,000$.

What exchanges our yETH will be listed?

We will not disclose the names all here, but we will be listed in at least 3 Dex, and 3 centralized exchanges in Quarter 2, 2021

When can we sell our eYTH?

All YETH tokens are distributed during the pre-sale will be locked in a trading contract for a period of more than 12 months and during the public sale is 6 months.

Would you like to earn many tokens and cryptocurrencies right now! ☞ CLICK HERE

Looking for more information…

☞ Website
☞ Explorer
☞ Source Code
☞ Social Channel
Message Board
☞ Documentation
☞ Coinmarketcap

Thank for visiting and reading this article! I’m highly appreciate your actions! Please share if you liked it!

#blockchain #bitcoin #crypto #fyeth.finance #yeth

Angelina roda

Angelina roda

1624219980

How to Buy NFT Art Finance Token - The EASIEST METHOD! DO NOT MISS!!! JUST IN 4 MINUTES

NFT Art Finance is currently one of the most popular cryptocurrencies right now on the market, so in today’s video, I will be showing you guys how to easily buy NFT Art Finance on your phone using the Trust Wallet application.
📺 The video in this post was made by More LimSanity
The origin of the article: https://www.youtube.com/watch?v=sKE6Pc_w1IE
🔺 DISCLAIMER: The article is for information sharing. The content of this video is solely the opinions of the speaker who is not a licensed financial advisor or registered investment advisor. Not investment advice or legal advice.
Cryptocurrency trading is VERY risky. Make sure you understand these risks and that you are responsible for what you do with your money
🔥 If you’re a beginner. I believe the article below will be useful to you ☞ What You Should Know Before Investing in Cryptocurrency - For Beginner
⭐ ⭐ ⭐The project is of interest to the community. Join to Get free ‘GEEK coin’ (GEEKCASH coin)!
☞ **-----CLICK HERE-----**⭐ ⭐ ⭐
Thanks for visiting and watching! Please don’t forget to leave a like, comment and share!

#bitcoin #blockchain #nft art finance token #token #buy nft art finance #how to buy nft art finance token - the easiest method!

David mr

David mr

1624312800

SPORE FINANCE PREDICTION - WHAT IS SPORE FINANCE & SPORE FINANCE ANALYSIS - SPORE FINANCE

SPORE FINANCE PREDICTION - WHAT IS SPORE FINANCE & SPORE FINANCE ANALYSIS - SPORE FINANCE

In this video, I talk about spore finance coin and give my spore finance prediction. I talk about the latest spore finance analysis & spore finance crypto coin that recently has been hit pretty hard in the last 24 hours. I go over what is spore finance and how many holders are on this new crypto coin spore finance.
📺 The video in this post was made by Josh’s Finance
The origin of the article: https://www.youtube.com/watch?v=qbPQvdxCtEI
🔺 DISCLAIMER: The article is for information sharing. The content of this video is solely the opinions of the speaker who is not a licensed financial advisor or registered investment advisor. Not investment advice or legal advice.
Cryptocurrency trading is VERY risky. Make sure you understand these risks and that you are responsible for what you do with your money
🔥 If you’re a beginner. I believe the article below will be useful to you ☞ What You Should Know Before Investing in Cryptocurrency - For Beginner
⭐ ⭐ ⭐The project is of interest to the community. Join to Get free ‘GEEK coin’ (GEEKCASH coin)!
☞ **-----CLICK HERE-----**⭐ ⭐ ⭐
Thanks for visiting and watching! Please don’t forget to leave a like, comment and share!

#bitcoin #blockchain #spore finance #what is spore finance #spore finance prediction - what is spore finance & spore finance analysis - spore finance #spore finance prediction

Royce  Reinger

Royce Reinger

1658068560

WordsCounted: A Ruby Natural Language Processor

WordsCounted

We are all in the gutter, but some of us are looking at the stars.

-- Oscar Wilde

WordsCounted is a Ruby NLP (natural language processor). WordsCounted lets you implement powerful tokensation strategies with a very flexible tokeniser class.

Features

  • Out of the box, get the following data from any string or readable file, or URL:
    • Token count and unique token count
    • Token densities, frequencies, and lengths
    • Char count and average chars per token
    • The longest tokens and their lengths
    • The most frequent tokens and their frequencies.
  • A flexible way to exclude tokens from the tokeniser. You can pass a string, regexp, symbol, lambda, or an array of any combination of those types for powerful tokenisation strategies.
  • Pass your own regexp rules to the tokeniser if you prefer. The default regexp filters special characters but keeps hyphens and apostrophes. It also plays nicely with diacritics (UTF and unicode characters): Bayrūt is treated as ["Bayrūt"] and not ["Bayr", "ū", "t"], for example.
  • Opens and reads files. Pass in a file path or a url instead of a string.

Installation

Add this line to your application's Gemfile:

gem 'words_counted'

And then execute:

$ bundle

Or install it yourself as:

$ gem install words_counted

Usage

Pass in a string or a file path, and an optional filter and/or regexp.

counter = WordsCounted.count(
  "We are all in the gutter, but some of us are looking at the stars."
)

# Using a file
counter = WordsCounted.from_file("path/or/url/to/my/file.txt")

.count and .from_file are convenience methods that take an input, tokenise it, and return an instance of WordsCounted::Counter initialized with the tokens. The WordsCounted::Tokeniser and WordsCounted::Counter classes can be used alone, however.

API

WordsCounted

WordsCounted.count(input, options = {})

Tokenises input and initializes a WordsCounted::Counter object with the resulting tokens.

counter = WordsCounted.count("Hello Beirut!")

Accepts two options: exclude and regexp. See Excluding tokens from the analyser and Passing in a custom regexp respectively.

WordsCounted.from_file(path, options = {})

Reads and tokenises a file, and initializes a WordsCounted::Counter object with the resulting tokens.

counter = WordsCounted.from_file("hello_beirut.txt")

Accepts the same options as .count.

Tokeniser

The tokeniser allows you to tokenise text in a variety of ways. You can pass in your own rules for tokenisation, and apply a powerful filter with any combination of rules as long as they can boil down into a lambda.

Out of the box the tokeniser includes only alpha chars. Hyphenated tokens and tokens with apostrophes are considered a single token.

#tokenise([pattern: TOKEN_REGEXP, exclude: nil])

tokeniser = WordsCounted::Tokeniser.new("Hello Beirut!").tokenise

# With `exclude`
tokeniser = WordsCounted::Tokeniser.new("Hello Beirut!").tokenise(exclude: "hello")

# With `pattern`
tokeniser = WordsCounted::Tokeniser.new("I <3 Beirut!").tokenise(pattern: /[a-z]/i)

See Excluding tokens from the analyser and Passing in a custom regexp for more information.

Counter

The WordsCounted::Counter class allows you to collect various statistics from an array of tokens.

#token_count

Returns the token count of a given string.

counter.token_count #=> 15

#token_frequency

Returns a sorted (unstable) two-dimensional array where each element is a token and its frequency. The array is sorted by frequency in descending order.

counter.token_frequency

[
  ["the", 2],
  ["are", 2],
  ["we",  1],
  # ...
  ["all", 1]
]

#most_frequent_tokens

Returns a hash where each key-value pair is a token and its frequency.

counter.most_frequent_tokens

{ "are" => 2, "the" => 2 }

#token_lengths

Returns a sorted (unstable) two-dimentional array where each element contains a token and its length. The array is sorted by length in descending order.

counter.token_lengths

[
  ["looking", 7],
  ["gutter",  6],
  ["stars",   5],
  # ...
  ["in",      2]
]

#longest_tokens

Returns a hash where each key-value pair is a token and its length.

counter.longest_tokens

{ "looking" => 7 }

#token_density([ precision: 2 ])

Returns a sorted (unstable) two-dimentional array where each element contains a token and its density as a float, rounded to a precision of two. The array is sorted by density in descending order. It accepts a precision argument, which must be a float.

counter.token_density

[
  ["are",     0.13],
  ["the",     0.13],
  ["but",     0.07 ],
  # ...
  ["we",      0.07 ]
]

#char_count

Returns the char count of tokens.

counter.char_count #=> 76

#average_chars_per_token([ precision: 2 ])

Returns the average char count per token rounded to two decimal places. Accepts a precision argument which defaults to two. Precision must be a float.

counter.average_chars_per_token #=> 4

#uniq_token_count

Returns the number of unique tokens.

counter.uniq_token_count #=> 13

Excluding tokens from the tokeniser

You can exclude anything you want from the input by passing the exclude option. The exclude option accepts a variety of filters and is extremely flexible.

  1. A space-delimited string. The filter will normalise the string.
  2. A regular expression.
  3. A lambda.
  4. A symbol that names a predicate method. For example :odd?.
  5. An array of any combination of the above.
tokeniser =
  WordsCounted::Tokeniser.new(
    "Magnificent! That was magnificent, Trevor."
  )

# Using a string
tokeniser.tokenise(exclude: "was magnificent")
# => ["that", "trevor"]

# Using a regular expression
tokeniser.tokenise(exclude: /trevor/)
# => ["magnificent", "that", "was", "magnificent"]

# Using a lambda
tokeniser.tokenise(exclude: ->(t) { t.length < 4 })
# => ["magnificent", "that", "magnificent", "trevor"]

# Using symbol
tokeniser = WordsCounted::Tokeniser.new("Hello! محمد")
tokeniser.tokenise(exclude: :ascii_only?)
# => ["محمد"]

# Using an array
tokeniser = WordsCounted::Tokeniser.new(
  "Hello! اسماءنا هي محمد، كارولينا، سامي، وداني"
)
tokeniser.tokenise(
  exclude: [:ascii_only?, /محمد/, ->(t) { t.length > 6}, "و"]
)
# => ["هي", "سامي", "وداني"]

Passing in a custom regexp

The default regexp accounts for letters, hyphenated tokens, and apostrophes. This means twenty-one is treated as one token. So is Mohamad's.

/[\p{Alpha}\-']+/

You can pass your own criteria as a Ruby regular expression to split your string as desired.

For example, if you wanted to include numbers, you can override the regular expression:

counter = WordsCounted.count("Numbers 1, 2, and 3", pattern: /[\p{Alnum}\-']+/)
counter.tokens
#=> ["numbers", "1", "2", "and", "3"]

Opening and reading files

Use the from_file method to open files. from_file accepts the same options as .count. The file path can be a URL.

counter = WordsCounted.from_file("url/or/path/to/file.text")

Gotchas

A hyphen used in leu of an em or en dash will form part of the token. This affects the tokeniser algorithm.

counter = WordsCounted.count("How do you do?-you are well, I see.")
counter.token_frequency

[
  ["do",   2],
  ["how",  1],
  ["you",  1],
  ["-you", 1], # WTF, mate!
  ["are",  1],
  # ...
]

In this example -you and you are separate tokens. Also, the tokeniser does not include numbers by default. Remember that you can pass your own regular expression if the default behaviour does not fit your needs.

A note on case sensitivity

The program will normalise (downcase) all incoming strings for consistency and filters.

Roadmap

Ability to open URLs

def self.from_url
  # open url and send string here after removing html
end

Are you using WordsCounted to do something interesting? Please tell me about it.

Gem Version 

RubyDoc documentation.

Demo

Visit this website for one example of what you can do with WordsCounted.


Contributors

See contributors.

Contributing

  1. Fork it
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create new Pull Request

Author: Abitdodgy
Source Code: https://github.com/abitdodgy/words_counted 
License: MIT license

#ruby #nlp 

Words Counted: A Ruby Natural Language Processor.

WordsCounted

We are all in the gutter, but some of us are looking at the stars.

-- Oscar Wilde

WordsCounted is a Ruby NLP (natural language processor). WordsCounted lets you implement powerful tokensation strategies with a very flexible tokeniser class.

Are you using WordsCounted to do something interesting? Please tell me about it.

 

Demo

Visit this website for one example of what you can do with WordsCounted.

Features

  • Out of the box, get the following data from any string or readable file, or URL:
    • Token count and unique token count
    • Token densities, frequencies, and lengths
    • Char count and average chars per token
    • The longest tokens and their lengths
    • The most frequent tokens and their frequencies.
  • A flexible way to exclude tokens from the tokeniser. You can pass a string, regexp, symbol, lambda, or an array of any combination of those types for powerful tokenisation strategies.
  • Pass your own regexp rules to the tokeniser if you prefer. The default regexp filters special characters but keeps hyphens and apostrophes. It also plays nicely with diacritics (UTF and unicode characters): Bayrūt is treated as ["Bayrūt"] and not ["Bayr", "ū", "t"], for example.
  • Opens and reads files. Pass in a file path or a url instead of a string.

Installation

Add this line to your application's Gemfile:

gem 'words_counted'

And then execute:

$ bundle

Or install it yourself as:

$ gem install words_counted

Usage

Pass in a string or a file path, and an optional filter and/or regexp.

counter = WordsCounted.count(
  "We are all in the gutter, but some of us are looking at the stars."
)

# Using a file
counter = WordsCounted.from_file("path/or/url/to/my/file.txt")

.count and .from_file are convenience methods that take an input, tokenise it, and return an instance of WordsCounted::Counter initialized with the tokens. The WordsCounted::Tokeniser and WordsCounted::Counter classes can be used alone, however.

API

WordsCounted

WordsCounted.count(input, options = {})

Tokenises input and initializes a WordsCounted::Counter object with the resulting tokens.

counter = WordsCounted.count("Hello Beirut!")

Accepts two options: exclude and regexp. See Excluding tokens from the analyser and Passing in a custom regexp respectively.

WordsCounted.from_file(path, options = {})

Reads and tokenises a file, and initializes a WordsCounted::Counter object with the resulting tokens.

counter = WordsCounted.from_file("hello_beirut.txt")

Accepts the same options as .count.

Tokeniser

The tokeniser allows you to tokenise text in a variety of ways. You can pass in your own rules for tokenisation, and apply a powerful filter with any combination of rules as long as they can boil down into a lambda.

Out of the box the tokeniser includes only alpha chars. Hyphenated tokens and tokens with apostrophes are considered a single token.

#tokenise([pattern: TOKEN_REGEXP, exclude: nil])

tokeniser = WordsCounted::Tokeniser.new("Hello Beirut!").tokenise

# With `exclude`
tokeniser = WordsCounted::Tokeniser.new("Hello Beirut!").tokenise(exclude: "hello")

# With `pattern`
tokeniser = WordsCounted::Tokeniser.new("I <3 Beirut!").tokenise(pattern: /[a-z]/i)

See Excluding tokens from the analyser and Passing in a custom regexp for more information.

Counter

The WordsCounted::Counter class allows you to collect various statistics from an array of tokens.

#token_count

Returns the token count of a given string.

counter.token_count #=> 15

#token_frequency

Returns a sorted (unstable) two-dimensional array where each element is a token and its frequency. The array is sorted by frequency in descending order.

counter.token_frequency

[
  ["the", 2],
  ["are", 2],
  ["we",  1],
  # ...
  ["all", 1]
]

#most_frequent_tokens

Returns a hash where each key-value pair is a token and its frequency.

counter.most_frequent_tokens

{ "are" => 2, "the" => 2 }

#token_lengths

Returns a sorted (unstable) two-dimentional array where each element contains a token and its length. The array is sorted by length in descending order.

counter.token_lengths

[
  ["looking", 7],
  ["gutter",  6],
  ["stars",   5],
  # ...
  ["in",      2]
]

#longest_tokens

Returns a hash where each key-value pair is a token and its length.

counter.longest_tokens

{ "looking" => 7 }

#token_density([ precision: 2 ])

Returns a sorted (unstable) two-dimentional array where each element contains a token and its density as a float, rounded to a precision of two. The array is sorted by density in descending order. It accepts a precision argument, which must be a float.

counter.token_density

[
  ["are",     0.13],
  ["the",     0.13],
  ["but",     0.07 ],
  # ...
  ["we",      0.07 ]
]

#char_count

Returns the char count of tokens.

counter.char_count #=> 76

#average_chars_per_token([ precision: 2 ])

Returns the average char count per token rounded to two decimal places. Accepts a precision argument which defaults to two. Precision must be a float.

counter.average_chars_per_token #=> 4

#uniq_token_count

Returns the number of unique tokens.

counter.uniq_token_count #=> 13

Excluding tokens from the tokeniser

You can exclude anything you want from the input by passing the exclude option. The exclude option accepts a variety of filters and is extremely flexible.

  1. A space-delimited string. The filter will normalise the string.
  2. A regular expression.
  3. A lambda.
  4. A symbol that names a predicate method. For example :odd?.
  5. An array of any combination of the above.
tokeniser =
  WordsCounted::Tokeniser.new(
    "Magnificent! That was magnificent, Trevor."
  )

# Using a string
tokeniser.tokenise(exclude: "was magnificent")
# => ["that", "trevor"]

# Using a regular expression
tokeniser.tokenise(exclude: /trevor/)
# => ["magnificent", "that", "was", "magnificent"]

# Using a lambda
tokeniser.tokenise(exclude: ->(t) { t.length < 4 })
# => ["magnificent", "that", "magnificent", "trevor"]

# Using symbol
tokeniser = WordsCounted::Tokeniser.new("Hello! محمد")
tokeniser.tokenise(exclude: :ascii_only?)
# => ["محمد"]

# Using an array
tokeniser = WordsCounted::Tokeniser.new(
  "Hello! اسماءنا هي محمد، كارولينا، سامي، وداني"
)
tokeniser.tokenise(
  exclude: [:ascii_only?, /محمد/, ->(t) { t.length > 6}, "و"]
)
# => ["هي", "سامي", "وداني"]

Passing in a custom regexp

The default regexp accounts for letters, hyphenated tokens, and apostrophes. This means twenty-one is treated as one token. So is Mohamad's.

/[\p{Alpha}\-']+/

You can pass your own criteria as a Ruby regular expression to split your string as desired.

For example, if you wanted to include numbers, you can override the regular expression:

counter = WordsCounted.count("Numbers 1, 2, and 3", pattern: /[\p{Alnum}\-']+/)
counter.tokens
#=> ["numbers", "1", "2", "and", "3"]

Opening and reading files

Use the from_file method to open files. from_file accepts the same options as .count. The file path can be a URL.

counter = WordsCounted.from_file("url/or/path/to/file.text")

Gotchas

A hyphen used in leu of an em or en dash will form part of the token. This affects the tokeniser algorithm.

counter = WordsCounted.count("How do you do?-you are well, I see.")
counter.token_frequency

[
  ["do",   2],
  ["how",  1],
  ["you",  1],
  ["-you", 1], # WTF, mate!
  ["are",  1],
  # ...
]

In this example -you and you are separate tokens. Also, the tokeniser does not include numbers by default. Remember that you can pass your own regular expression if the default behaviour does not fit your needs.

A note on case sensitivity

The program will normalise (downcase) all incoming strings for consistency and filters.

Roadmap

Ability to open URLs

def self.from_url
  # open url and send string here after removing html
end

Contributors

See contributors.

Contributing

  1. Fork it
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create new Pull Request

Author: abitdodgy
Source code: https://github.com/abitdodgy/words_counted
License: MIT license

#ruby  #ruby-on-rails