What is Brickken (BKN) | What is Brickken token | What is BKN token

In this article, we'll discuss information about the Brickken project and BKN token. What is Brickken (BKN) | What is Brickken token | What is BKN token?

Brickken is creating a dApp (decentralized application) which provides the infrastructure needed for companies to bring their assets on-chain and self-fund themselves via Security Token Offerings (STO), and for DAOs to be able to operate in a legally compliant environment. Our goal is to bridge offline to online, and web2 to web3 to prepare the world for a new fully tokenized reality.

Brickken’s protocol provides the user with an easy to use interface that is best-in-class, allowing to onboard companies, DAOs, investors, and service providers into a same ecosystem.

Brickken’s decentralized management system opens new opportunities for funding, governance, measure performance, and manage treasuries.

Values

Autonomy: Brickken’s dApp provides the technical infrastructure for companies to operate in a Web3 environment and create their STOs as form of funding. It allows any business from any walk of life to move from ambition to action and succeed in realizing their projects.
Simplicity: Brickken’s dApp is being built with emphasis on simplicity, to make the tokenization process as simple and accessible as possible.
Compliance: Brickken’s dApp will help companies transform their assets such as shares into tokens, and issue their own STOs as per the applicable legislation in their host State, whilst complying with the local regulation regarding issuance of securities and asset management. The same infrastructure will provide DAOs with the needed tools to operate and be managed.
Democracy: Brickken’s dApp paves the way to open illiquid markets to the world, creating a solution to democratize fundraising and bridge the gap between legacy finance, retail investing and the decentralized Cryptoverse resulting in new investment opportunities.

The ecosystem

1. Security token offerings

Utilizing a decentralized platform presents two prominent challenges:

Technological: it connects the dApp’s back end which runs completely on a public blockchain through smart contracts. The smart contracts must be flawless to ensure the protection of issuers of tokens and their investors.
Legal/regulatory: the objective is to issue STOs in a compliant manner. Investors must also pass a KYC process to comply with regulatory and anti-money laundering regulations.

When dealing with standard securities, the ownership information of the investment product is recorded in a certificate which can take the form of a simple PDF. With a security token, the information is stored into an immutable blockchain and instead of a certificate being issued, a token is.

All countries have very precise and extensive regulations in relation to what securities are, how they must be issued, who can participate, who can buy them, and what protection investors are afforded.

Brickken aims to create a decentralized uniform protocol of security token issuance.

The goal is for any issuer of security tokens to use Brickken’s technology, and for this issuer to:

Comply with local regulations.
Comply with the specifications of the issuance itself (the what).
Comply with the process of how selling the security tokens may occur and by whom (the how).
Slow legally compliant transfer of security tokens to occur in secondary markets (the where).

Additionally, it is important to consider that while the biggest complexity lies in providing a solution that is compliant from a regulatory and technological point of view, the financial structure and tokenomics of the security token issuance must adhere to the end goal of the project.

In this sense, the dApp must allow the issuer to establish what are the hard and the soft caps, what is the term or maturity of the loan in case of issuance of debt, or what is the shareholding allocation in the case of tokenized shares.

2. Security token smart contracts

Smart contracts will be utilized to create two entities: the ERC2011 BKN utility token and an STO factory. The latter will deploy ERC20 dedicated STO tokens and escrow contracts for each STO that is issued through Brickken’s dApp.

The BKN token will be the utility token associated with the dApp platform. With it, promoters can issue their own STOs.

The BKN utility token will be accessible via a Uniswap pool12. Promoters will need to acquire BKN to use the dApp. BKN is then used when performing STO activities.

Brickken aims to achieve the highest level of security, using audited libraries and smart contracts that follow the best practices to reduce attack vectors and possible exploits.

3. The BKN utility token

The BKN utility token will have two fundamentally different and well-defined stages: the pre-issuance and the public sale.

The objective of the pre-issuance stage is to allocate BKN utility tokens to future Security Token Offering (STO) issuers and different groups of people who trust in Brickken’s vision.

The public sale will be conducted after the pre-issuance period has ended.

At this stage, the BKN utility token will be placed through an IEO and subsequently in a Uniswap pool, which will be pre-funded with USDC (a stablecoin) and BKN to enhance possibilities of the dApp.

Brickken will need to fund the pool to establish a fixed starting price, since automated market makers, and in particular Uniswap, use the Constant Product Formula to establish the price based on BKN/USDC pair funds deposited in the pool.

Naturally, both x and y amounts (BKN and USDC) must be funded. The amount to be supplied is determined by the target starting price.

The utility tokens ERC20 contract will implement several functional advantages such as:

Representation of voting powers depending on the pro-rata holding of BKN.
STO issuers will hold BKN as collateral and there will be a system of rewards and penalties for issuers that fulfill the obligations established in their respective STO white papers. These rewards will come in the form of staking and slashing.
Compliance with many Ethereum improvement proposals such as EIP712 and EIP165.
Brickken will propose the creation of a DAO (Decentralized Autonomous Organization), which will govern decision making on matters such as protocol upgrades, allocation of social funds, and governance among others.
Serve as a payment token to third parties belonging to Brickken’s Experts Ecosystem

Tokenization

Any asset or business can be broken down in fractional parts that retain the form of tokens, with equal rights and values, that can be purchased by anyone, anywhere, at any time.

Brickken offers a market leading, legally compliant, decentralized platform to perform Security Token Offerings (STO´s) and investment management, making a secure, transparent, convenient solution in which to raise funds through tokenization.

Architecture

Brickken’s dApp architecture is modular, based on microservices that connect to each other to facilitate the usage, upgradeability, and maintenance of the protocol.

brickken_dapp

FRONT-END: consist in micro front-ends that render components for a set of specific routes it is more performant than monoliths front-end.. the front-end will integrate MetaMask and an API service. The MetaMask integration is required to allow users to interact directly with the blockchain and protocol contracts, while the API service will handle KYC management, user logins, registrations, and general operations. The user has different views: he/she first has a smart contract wizard where the issuer user can deploy Security Tokens; once the company token is deployed and the smart contract is created, he/she will be able to access the dashboard where the company can be managed onchain, create pools, STOs, customize the launchpad for STOs, reviewanalytics around offerings, among others.
BACK-END: The back-end will consist of a set of lambdas functions that serve the dashboard app, but also expose the tokenization service without the need for a dashboard. For this, the transactions that are sent through an API integration will use Infura instead of MetaMask to send the transactions to the blockchain.
KYC SERVICE: The KYC service solution will connect to our existing Lambdas system. Acceptance and rejection of requests can be executed with the API or through the dashboard.
DATABASE: The database will store useful information for the functionality that Brickken proposes to build (users that might register their emails, metadata’s, transactions, etc.).
SMART CONTRACTS: The smart contracts will be made up of an ERC20 token contract (BKN), and a smart contract that serve as a factory for two other smart contracts;
- The escrow contract is where investors deposit money until the STO is finalized.
- The ERC20 token contract will represent STO specific tokens. The escrow contract will also be the one that issuers use to deposit the dividends/ interests that serves as revenue (yield) for the investors.

Additional challenges

Achieving scalability and low-cost bases in public blockchain networks

Existing solutions must deal with the lack of scalability and costs associated with public blockchain networks such as the Ethereum mainnet. For this reason, the solution adopted in many cases is to use a private blockchain, renouncing decentralization as an objective.

Decentralization is a fundamental principle that governs Brickken’s existence, given the advancement of scalability and cost solutions, we are comfortable building on Ethereum’s mainnet.

To overcome this challenge, a solution is proposed based on the following flow:

brickken_flow

The issuer will buy BKN from a Uniswap pool that Brickken will create during the public sale of the BKN utility token.
Once the issuer receives BKN utility tokens, the issuer can initiate a tokenization specifying the asset to be tokenized alongside the terms and conditions of the tokenization (purpose, place of issuance, financial terms, tokenomics, deadlines, etc.).
The result of this process will be two smart contracts: an STO token and an escrow contract. Used BKN will be stacked in the escrow contract while the STO is running.
The issuer must establish what is the hard cap and soft cap, but also when the STO starts and when it ends.
Investors will be able to invest in STOs with any cryptocurrency. Investors’ money will be automatically converted to the stablecoin USDC. This is to avoid wide fluctuations of value in the short period of time that lasts between deposits being made to the finalization of the STO.
If the escrow reaches the soft or hard-cap the tokenization will be considered successful. The escrow will release the capital to the issuer and the security tokens to the investors. In a successful STO, the issuer will be able to withdraw the stake but if he decides to keep it staked it will be earning rewards over successful dividend payments. At the same time, bad issuers can have their stakes slashed if they don’t meet dividend payments.

Investors will accrue interest and/or dividends on their acquired security tokens, thereafter, according to the terms and conditions of the STO, the issuer must deposit the accrued interests or dividends in the escrow contract. This objective will be achieved through the following flow:

Technical approach to KYC implementation

The next technological challenge is related to legal aspects. For any user to be able to invest in an asset tokenization, it is a legal requirement to successfully pass a KYC process. Brickken’s blockchain will comply with global data protection standards and applicable legislations, thus, cannot handle the personal data of users required for the on-chain KYC processes.

Therefore, Brickken will offer a hybrid solution where the KYC processes will be completed off-chain (outside the blockchain network); users are manually registered in a whitelist in the protocol’s smart contracts. Therefore, no personal data of the user is stored in the blockchain, but its registration will trigger whitelisting, and will pass through the blockchain using smart contracts to accept users who have previously passed their KYC successfully.

To overcome this challenge, a solution based on the following flow is proposed:

Issuers/Promoters send their KYC requests to Brickken. Brickken will have a white/blacklist to automatically resolve requests. Once resolved the issuer can now use the platform.
The issuer is now validated and can start accepting or rejecting requests from investors.
An unknown investor sends a KYC request to participate in an STO. Request is accepted/rejected by the issuer of that specific STO. A transaction is sent to the blockchain to whitelist the accepted investor in receiving STO tokens.
The investor is now a validated user. The investor wants now to transfer the STO’s token to an unknown buyer.
The unknown buyer submits a KYC request to the same issuer.
The unknown buyer has the KYC request accepted and it’s now whitelisted to receive the STO’s token.
The validated buyer receives the STO tokens.

Token Economy

Initial Coin Distribution

The issuance will have an initial limit of BKN [150,000,000].

Of this amount, the liquidity pool which will be created will be made by depositing 12,000,000 (8%) into the Uniswap pool to ensure the stability of the token; the pool will be funded with USDC. USDC is a stablecoin and it will come from a liquidity vault created from the value generated by the protocol from its business model.

The initial open price will be set by the relation between the 12.000.000 BKN coming from tokenization fee and the amount of money raised by all the companies brought on chain by the protocol. This implies that the price ratio is going to be determined by the added value that the protocol brings to the ecosystem at the time of closing the 12.000.000 BKN.

Uniswap is an automated liquidity protocol powered by a Constant Product Formula and implemented in a system of non-upgradeable smart contracts on the Ethereum blockchain. This removes the need for trusted intermediaries, prioritizing decentralization, censorship-resistance, and security. Uniswap is open-source software licensed under the GPL.

After the funding of the liquidity pool a percentage of the liquidity vault would be used to fund proposals on the Brickken DAO adding more stakeholders to the equation and allowing the protocol to grow in a decentralized way.

Social and DAO

Tokenization will change the world as we know it. One of the main considerations of STOs is to allow smaller, less fortunate individuals and businesses, to access capital markets in a less restricted approach. For instance, certain undeveloped countries may lack the appropriate financial infrastructure to gain access to funding, or countries with questionable governments may suffer from corruption or other nation state related risks that may result in investors rejecting investment propositions from these countries.

To enable individuals and businesses to have the same access to capital markets as other more developed countries, Brickken is considering allocating a share of its profits to allow smaller individuals and businesses to obtain a discount on the cost of tokenization.

As mentioned, Brickken intends to develop a DAO. The purpose of the DAO will be to provide community members with decisional powers on protocol upgrades. For example, to incentivise the allocation of social funds to fund STOs which are governed by ESG parameters, general governance, and protocol upgrades.

The goal is to be as decentralized as possible with a major emphasis on involving the community, which will be crucial to the success of the project.

Incentivising the self-sustainability of the ecosystem by the use of BKN as the vehicle of investment

It is Brickken’s purpose to design and develop a self-regulated STO ecosystem. We acknowledge that certain limitations arise from full decentralization. Among others, it is possible that certain STO issuers intentionally try to misuse the ecosystem itself. Such decentralization makes it difficult to actually control who can and cannot access the ecosystem. As a result, Brickken will develop an embedded system that will carry out certain actions automatically and autonomously to safeguard investors’ interests.

Security Token Offerings (STOs) will have a collateral mechanism by which the BKN received by issuers carrying a tokenization from investors will be deposited in the escrow account taking the shape of stake. The escrow account will mint or slash the rewards which will be added or withdrawn to the balance of BKN Depending on the amount of BKN received and the behavior of the issuer.

The STO factory will incorporate a reward/penalty system by which Good Actors will receive staking in the form of BKN that could be sold or used to deploy additional STOs or services payable in BKN. On the other hand, Bad Actors will be penalized seeing their BKN deposits slashed and will be included on a Blacklist.

Allocation

	Allocation %	Allocation BKN
Community Sale	16,67	25.000.000
Warchest	26,67	40.000.000
Team	16,67	25.000.000
Liquidity Pool	8,00	12.000.000
Advisors	1,67	2.500.000
Airdrop	1,00	1.500.000
Strategic Sale	16,67	25.000.000
Public Sale 1	5,33	8.000.000
Public Sale 2	5,33	8.000.000
Community Incentives	2,00	3.000.000
Total	100	150.000.000

How and Where to Buy token?

You will have to first buy one of the major cryptocurrencies, usually Bitcoin (BTC), Ethereum (ETH), Tether (USDT), Binance (BNB)…

We will use Binance Exchange here as it is one of the largest crypto exchanges that accept fiat deposits.

Once you finished the KYC process. You will be asked to add a payment method. Here you can either choose to provide a credit/debit card or use a bank transfer, and buy one of the major cryptocurrencies, usually either Bitcoin (BTC), Ethereum (ETH), Tether (USDT), Binance (BNB)…

☞ SIGN UP ON BINANCE

Once finished you will then need to make a BTC/ETH/USDT/BNB deposit to the exchange from Binance depending on the available market pairs. After the deposit is confirmed you may then purchase token from the exchange.

Top exchanges for token-coin trading. Follow instructions and make unlimited money

☞ Binance ☞ Poloniex ☞ Bitfinex ☞ Huobi ☞ MXC ☞ ProBIT ☞ Gate.io

🔥 If you’re a beginner. I believe the article below will be useful to you ☞ What You Should Know Before Investing in Cryptocurrency - For Beginner

🔺DISCLAIMER: The Information in the post isn’t financial advice, is intended FOR GENERAL INFORMATION PURPOSES ONLY. Trading Cryptocurrency is VERY risky. Make sure you understand these risks and that you are responsible for what you do with your money.

I hope this post will help you. Don't forget to leave a like, comment and sharing it with others. Thank you!

#bitcoin #cryptocurrency #token #coin

What is GEEK

Buddha Community

What is Brickken (BKN) | What is Brickken token | What is BKN token

Margo Margo

1671913486

crypto currency is really very popular now and many people can now easily start investing in it, and you can also try to play on the crypt. you only need to find a suitable site that you will trust. Here I can advise you an article where there is a list of verified sites. i'm sure you'll like them

Mohammed Richards

1659601560

Words Counted: A Ruby Natural Language Processor.

WordsCounted

We are all in the gutter, but some of us are looking at the stars.
-- Oscar Wilde

WordsCounted is a Ruby NLP (natural language processor). WordsCounted lets you implement powerful tokensation strategies with a very flexible tokeniser class.

Are you using WordsCounted to do something interesting? Please tell me about it.

Demo

Visit this website for one example of what you can do with WordsCounted.

Features

Out of the box, get the following data from any string or readable file, or URL:
- Token count and unique token count
- Token densities, frequencies, and lengths
- Char count and average chars per token
- The longest tokens and their lengths
- The most frequent tokens and their frequencies.
A flexible way to exclude tokens from the tokeniser. You can pass a string, regexp, symbol, lambda, or an array of any combination of those types for powerful tokenisation strategies.
Pass your own regexp rules to the tokeniser if you prefer. The default regexp filters special characters but keeps hyphens and apostrophes. It also plays nicely with diacritics (UTF and unicode characters): Bayrūt is treated as ["Bayrūt"] and not ["Bayr", "ū", "t"], for example.
Opens and reads files. Pass in a file path or a url instead of a string.

Installation

Add this line to your application's Gemfile:

gem 'words_counted'

And then execute:

$ bundle

Or install it yourself as:

$ gem install words_counted

Usage

Pass in a string or a file path, and an optional filter and/or regexp.

counter = WordsCounted.count(
  "We are all in the gutter, but some of us are looking at the stars."
)

# Using a file
counter = WordsCounted.from_file("path/or/url/to/my/file.txt")

.count and .from_file are convenience methods that take an input, tokenise it, and return an instance of WordsCounted::Counter initialized with the tokens. The WordsCounted::Tokeniser and WordsCounted::Counter classes can be used alone, however.

API

WordsCounted

WordsCounted.count(input, options = {})

Tokenises input and initializes a WordsCounted::Counter object with the resulting tokens.

counter = WordsCounted.count("Hello Beirut!")

Accepts two options: exclude and regexp. See Excluding tokens from the analyser and Passing in a custom regexp respectively.

WordsCounted.from_file(path, options = {})

Reads and tokenises a file, and initializes a WordsCounted::Counter object with the resulting tokens.

counter = WordsCounted.from_file("hello_beirut.txt")

Accepts the same options as .count.

Tokeniser

The tokeniser allows you to tokenise text in a variety of ways. You can pass in your own rules for tokenisation, and apply a powerful filter with any combination of rules as long as they can boil down into a lambda.

Out of the box the tokeniser includes only alpha chars. Hyphenated tokens and tokens with apostrophes are considered a single token.

#tokenise([pattern: TOKEN_REGEXP, exclude: nil])

tokeniser = WordsCounted::Tokeniser.new("Hello Beirut!").tokenise

# With `exclude`
tokeniser = WordsCounted::Tokeniser.new("Hello Beirut!").tokenise(exclude: "hello")

# With `pattern`
tokeniser = WordsCounted::Tokeniser.new("I <3 Beirut!").tokenise(pattern: /[a-z]/i)

See Excluding tokens from the analyser and Passing in a custom regexp for more information.

Counter

The WordsCounted::Counter class allows you to collect various statistics from an array of tokens.

#token_count

Returns the token count of a given string.

counter.token_count #=> 15

#token_frequency

Returns a sorted (unstable) two-dimensional array where each element is a token and its frequency. The array is sorted by frequency in descending order.

counter.token_frequency

[
  ["the", 2],
  ["are", 2],
  ["we",  1],
  # ...
  ["all", 1]
]

#most_frequent_tokens

Returns a hash where each key-value pair is a token and its frequency.

counter.most_frequent_tokens

{ "are" => 2, "the" => 2 }

#token_lengths

Returns a sorted (unstable) two-dimentional array where each element contains a token and its length. The array is sorted by length in descending order.

counter.token_lengths

[
  ["looking", 7],
  ["gutter",  6],
  ["stars",   5],
  # ...
  ["in",      2]
]

#longest_tokens

Returns a hash where each key-value pair is a token and its length.

counter.longest_tokens

{ "looking" => 7 }

#token_density([ precision: 2 ])

Returns a sorted (unstable) two-dimentional array where each element contains a token and its density as a float, rounded to a precision of two. The array is sorted by density in descending order. It accepts a precision argument, which must be a float.

counter.token_density

[
  ["are",     0.13],
  ["the",     0.13],
  ["but",     0.07 ],
  # ...
  ["we",      0.07 ]
]

#char_count

Returns the char count of tokens.

counter.char_count #=> 76

#average_chars_per_token([ precision: 2 ])

Returns the average char count per token rounded to two decimal places. Accepts a precision argument which defaults to two. Precision must be a float.

counter.average_chars_per_token #=> 4

#uniq_token_count

Returns the number of unique tokens.

counter.uniq_token_count #=> 13

Excluding tokens from the tokeniser

You can exclude anything you want from the input by passing the exclude option. The exclude option accepts a variety of filters and is extremely flexible.

A space-delimited string. The filter will normalise the string.
A regular expression.
A lambda.
A symbol that names a predicate method. For example :odd?.
An array of any combination of the above.

tokeniser =
  WordsCounted::Tokeniser.new(
    "Magnificent! That was magnificent, Trevor."
  )

# Using a string
tokeniser.tokenise(exclude: "was magnificent")
# => ["that", "trevor"]

# Using a regular expression
tokeniser.tokenise(exclude: /trevor/)
# => ["magnificent", "that", "was", "magnificent"]

# Using a lambda
tokeniser.tokenise(exclude: ->(t) { t.length < 4 })
# => ["magnificent", "that", "magnificent", "trevor"]

# Using symbol
tokeniser = WordsCounted::Tokeniser.new("Hello! محمد")
tokeniser.tokenise(exclude: :ascii_only?)
# => ["محمد"]

# Using an array
tokeniser = WordsCounted::Tokeniser.new(
  "Hello! اسماءنا هي محمد، كارولينا، سامي، وداني"
)
tokeniser.tokenise(
  exclude: [:ascii_only?, /محمد/, ->(t) { t.length > 6}, "و"]
)
# => ["هي", "سامي", "وداني"]

Passing in a custom regexp

The default regexp accounts for letters, hyphenated tokens, and apostrophes. This means twenty-one is treated as one token. So is Mohamad's.

/[\p{Alpha}\-']+/

You can pass your own criteria as a Ruby regular expression to split your string as desired.

For example, if you wanted to include numbers, you can override the regular expression:

counter = WordsCounted.count("Numbers 1, 2, and 3", pattern: /[\p{Alnum}\-']+/)
counter.tokens
#=> ["numbers", "1", "2", "and", "3"]

Opening and reading files

Use the from_file method to open files. from_file accepts the same options as .count. The file path can be a URL.

counter = WordsCounted.from_file("url/or/path/to/file.text")

Gotchas

A hyphen used in leu of an em or en dash will form part of the token. This affects the tokeniser algorithm.

counter = WordsCounted.count("How do you do?-you are well, I see.")
counter.token_frequency

[
  ["do",   2],
  ["how",  1],
  ["you",  1],
  ["-you", 1], # WTF, mate!
  ["are",  1],
  # ...
]

In this example -you and you are separate tokens. Also, the tokeniser does not include numbers by default. Remember that you can pass your own regular expression if the default behaviour does not fit your needs.

A note on case sensitivity

The program will normalise (downcase) all incoming strings for consistency and filters.

Roadmap

Ability to open URLs

def self.from_url
  # open url and send string here after removing html
end

Contributors

See contributors.

Contributing

Fork it
Create your feature branch (git checkout -b my-new-feature)
Commit your changes (git commit -am 'Add some feature')
Push to the branch (git push origin my-new-feature)
Create new Pull Request

Author: abitdodgy
Source code: https://github.com/abitdodgy/words_counted
License: MIT license

#ruby #ruby-on-rails

Royce Reinger

1658068560

WordsCounted: A Ruby Natural Language Processor

WordsCounted

We are all in the gutter, but some of us are looking at the stars.
-- Oscar Wilde

WordsCounted is a Ruby NLP (natural language processor). WordsCounted lets you implement powerful tokensation strategies with a very flexible tokeniser class.

Features

Out of the box, get the following data from any string or readable file, or URL:
- Token count and unique token count
- Token densities, frequencies, and lengths
- Char count and average chars per token
- The longest tokens and their lengths
- The most frequent tokens and their frequencies.
A flexible way to exclude tokens from the tokeniser. You can pass a string, regexp, symbol, lambda, or an array of any combination of those types for powerful tokenisation strategies.
Pass your own regexp rules to the tokeniser if you prefer. The default regexp filters special characters but keeps hyphens and apostrophes. It also plays nicely with diacritics (UTF and unicode characters): Bayrūt is treated as ["Bayrūt"] and not ["Bayr", "ū", "t"], for example.
Opens and reads files. Pass in a file path or a url instead of a string.

Installation

Add this line to your application's Gemfile:

gem 'words_counted'

And then execute:

$ bundle

Or install it yourself as:

$ gem install words_counted

Usage

Pass in a string or a file path, and an optional filter and/or regexp.

counter = WordsCounted.count(
  "We are all in the gutter, but some of us are looking at the stars."
)

# Using a file
counter = WordsCounted.from_file("path/or/url/to/my/file.txt")

.count and .from_file are convenience methods that take an input, tokenise it, and return an instance of WordsCounted::Counter initialized with the tokens. The WordsCounted::Tokeniser and WordsCounted::Counter classes can be used alone, however.

API

WordsCounted

WordsCounted.count(input, options = {})

Tokenises input and initializes a WordsCounted::Counter object with the resulting tokens.

counter = WordsCounted.count("Hello Beirut!")

Accepts two options: exclude and regexp. See Excluding tokens from the analyser and Passing in a custom regexp respectively.

WordsCounted.from_file(path, options = {})

Reads and tokenises a file, and initializes a WordsCounted::Counter object with the resulting tokens.

counter = WordsCounted.from_file("hello_beirut.txt")

Accepts the same options as .count.

Tokeniser

The tokeniser allows you to tokenise text in a variety of ways. You can pass in your own rules for tokenisation, and apply a powerful filter with any combination of rules as long as they can boil down into a lambda.

Out of the box the tokeniser includes only alpha chars. Hyphenated tokens and tokens with apostrophes are considered a single token.

#tokenise([pattern: TOKEN_REGEXP, exclude: nil])

tokeniser = WordsCounted::Tokeniser.new("Hello Beirut!").tokenise

# With `exclude`
tokeniser = WordsCounted::Tokeniser.new("Hello Beirut!").tokenise(exclude: "hello")

# With `pattern`
tokeniser = WordsCounted::Tokeniser.new("I <3 Beirut!").tokenise(pattern: /[a-z]/i)

See Excluding tokens from the analyser and Passing in a custom regexp for more information.

Counter

The WordsCounted::Counter class allows you to collect various statistics from an array of tokens.

#token_count

Returns the token count of a given string.

counter.token_count #=> 15

#token_frequency

Returns a sorted (unstable) two-dimensional array where each element is a token and its frequency. The array is sorted by frequency in descending order.

counter.token_frequency

[
  ["the", 2],
  ["are", 2],
  ["we",  1],
  # ...
  ["all", 1]
]

#most_frequent_tokens

Returns a hash where each key-value pair is a token and its frequency.

counter.most_frequent_tokens

{ "are" => 2, "the" => 2 }

#token_lengths

Returns a sorted (unstable) two-dimentional array where each element contains a token and its length. The array is sorted by length in descending order.

counter.token_lengths

[
  ["looking", 7],
  ["gutter",  6],
  ["stars",   5],
  # ...
  ["in",      2]
]

#longest_tokens

Returns a hash where each key-value pair is a token and its length.

counter.longest_tokens

{ "looking" => 7 }

#token_density([ precision: 2 ])

Returns a sorted (unstable) two-dimentional array where each element contains a token and its density as a float, rounded to a precision of two. The array is sorted by density in descending order. It accepts a precision argument, which must be a float.

counter.token_density

[
  ["are",     0.13],
  ["the",     0.13],
  ["but",     0.07 ],
  # ...
  ["we",      0.07 ]
]

#char_count

Returns the char count of tokens.

counter.char_count #=> 76

#average_chars_per_token([ precision: 2 ])

Returns the average char count per token rounded to two decimal places. Accepts a precision argument which defaults to two. Precision must be a float.

counter.average_chars_per_token #=> 4

#uniq_token_count

Returns the number of unique tokens.

counter.uniq_token_count #=> 13

Excluding tokens from the tokeniser

You can exclude anything you want from the input by passing the exclude option. The exclude option accepts a variety of filters and is extremely flexible.

A space-delimited string. The filter will normalise the string.
A regular expression.
A lambda.
A symbol that names a predicate method. For example :odd?.
An array of any combination of the above.

tokeniser =
  WordsCounted::Tokeniser.new(
    "Magnificent! That was magnificent, Trevor."
  )

# Using a string
tokeniser.tokenise(exclude: "was magnificent")
# => ["that", "trevor"]

# Using a regular expression
tokeniser.tokenise(exclude: /trevor/)
# => ["magnificent", "that", "was", "magnificent"]

# Using a lambda
tokeniser.tokenise(exclude: ->(t) { t.length < 4 })
# => ["magnificent", "that", "magnificent", "trevor"]

# Using symbol
tokeniser = WordsCounted::Tokeniser.new("Hello! محمد")
tokeniser.tokenise(exclude: :ascii_only?)
# => ["محمد"]

# Using an array
tokeniser = WordsCounted::Tokeniser.new(
  "Hello! اسماءنا هي محمد، كارولينا، سامي، وداني"
)
tokeniser.tokenise(
  exclude: [:ascii_only?, /محمد/, ->(t) { t.length > 6}, "و"]
)
# => ["هي", "سامي", "وداني"]

Passing in a custom regexp

The default regexp accounts for letters, hyphenated tokens, and apostrophes. This means twenty-one is treated as one token. So is Mohamad's.

/[\p{Alpha}\-']+/

You can pass your own criteria as a Ruby regular expression to split your string as desired.

For example, if you wanted to include numbers, you can override the regular expression:

counter = WordsCounted.count("Numbers 1, 2, and 3", pattern: /[\p{Alnum}\-']+/)
counter.tokens
#=> ["numbers", "1", "2", "and", "3"]

Opening and reading files

Use the from_file method to open files. from_file accepts the same options as .count. The file path can be a URL.

counter = WordsCounted.from_file("url/or/path/to/file.text")

Gotchas

A hyphen used in leu of an em or en dash will form part of the token. This affects the tokeniser algorithm.

counter = WordsCounted.count("How do you do?-you are well, I see.")
counter.token_frequency

[
  ["do",   2],
  ["how",  1],
  ["you",  1],
  ["-you", 1], # WTF, mate!
  ["are",  1],
  # ...
]

In this example -you and you are separate tokens. Also, the tokeniser does not include numbers by default. Remember that you can pass your own regular expression if the default behaviour does not fit your needs.

A note on case sensitivity

The program will normalise (downcase) all incoming strings for consistency and filters.

Roadmap

Ability to open URLs

def self.from_url
  # open url and send string here after removing html
end

Are you using WordsCounted to do something interesting? Please tell me about it.

RubyDoc documentation.

Demo

Visit this website for one example of what you can do with WordsCounted.

Contributors

See contributors.

Contributing

Fork it
Create your feature branch (git checkout -b my-new-feature)
Commit your changes (git commit -am 'Add some feature')
Push to the branch (git push origin my-new-feature)
Create new Pull Request

Author: Abitdodgy
Source Code: https://github.com/abitdodgy/words_counted
License: MIT license

#ruby #nlp

aaron silva

1622197808

SafeMoon Clone | Create A DeFi Token Like SafeMoon | DeFi token like SafeMoon

SafeMoon is a decentralized finance (DeFi) token. This token consists of RFI tokenomics and auto-liquidity generating protocol. A DeFi token like SafeMoon has reached the mainstream standards under the Binance Smart Chain. Its success and popularity have been immense, thus, making the majority of the business firms adopt this style of cryptocurrency as an alternative.

A DeFi token like SafeMoon is almost similar to the other crypto-token, but the only difference being that it charges a 10% transaction fee from the users who sell their tokens, in which 5% of the fee is distributed to the remaining SafeMoon owners. This feature rewards the owners for holding onto their tokens.

Read More @ https://bit.ly/3oFbJoJ

#create a defi token like safemoon #defi token like safemoon #safemoon token #safemoon token clone #defi token

Coin Introduction

1671444553