Extending Context Length of Falcon 40B Model"

This repository contains sample code to make it easier to extend the context length of the Falcon 40B model. Please read the blog post for further instructions.

This code may increase the context length of the Falcon-40b-sft-top-1-560. Before using this code, or the Falcon model, you should perform your own independent assessment, and take measures to ensure that your use would comply with your own specific quality control practices and standards, and that your use would comply with the local rules, laws, regulations, licenses and terms that apply to you, and your content. The author of this code does not make any representations, warranties or guarantees that your use of this code or the model will result in a particular outcome or result.

Getting Started

See the medium blog post:



This code is released under the Apache 2.0 software license


This code is a modification of the layers.py released by HuggingFace under an Apache 2.0 license in the following location:


Download Details:

Author: awslabs

Official Github: https://github.com/awslabs/extending-the-context-length-of-open-source-llms 

License: Apache-2.0 license

#data #data-analysis #data-science #aws 

Extending Context Length of Falcon 40B Model"
2.00 GEEK