Guides, Papers, Lecture, Notebooks & Resources for Prompt Engineering

Prompt Engineering Guide

Prompt engineering is a relatively new discipline for developing and optimizing prompts to efficiently use language models (LMs) for a wide variety of applications and research topics. Prompt engineering skills help to better understand the capabilities and limitations of large language models (LLMs). Researchers use prompt engineering to improve the capacity of LLMs on a wide range of common and complex tasks such as question answering and arithmetic reasoning. Developers use prompt engineering to design robust and effective prompting techniques that interface with LLMs and other tools.

Motivated by the high interest in developing with LLMs, we have created this new prompt engineering guide that contains all the latest papers, learning guides, lectures, references, and tools related to prompt engineering.

Happy Prompting!


The following are a set of guides on prompt engineering developed by us. Guides are work in progress.

If you are using the guide for your work, please cite us as follows:

author = {Saravia, Elvis},
journal = {},
month = {12},
title = {{Prompt Engineering Guide}},
year = {2022}

Feel free to open a PR if you think something is missing here. Always welcome feedback and suggestions. Just open an issue!

Announcements / Updates

  • 🎉 We have launched new web version of the guide here
  • 🎓 Partnered with Sphere to deliver a new course on Prompt Engineering for LLMs
  • 💬 New ChatGPT prompt engineering guide coming soon!
  • 🔥 We reached #1 on Hacker News on 21 Feb 2023
  • 🎉 The Prompt Engineering Lecture went live here
  • 🎓 We're creating a set of comprehensive guides here

Join our Discord

Follow us on Twitter

Subscribe to our Newsletter


We have published a 1 hour lecture that provides a comprehensive overview of prompting techniques, applications, and tools.

Download Details:

Author: Dair-ai
Source Code: 
License: MIT license

#deeplearning #engineering #jupyternotebook #notebooks 

Guides, Papers, Lecture, Notebooks & Resources for Prompt Engineering
Royce  Reinger

Royce Reinger


MachineLearningNotebooks: Azure Machine Learning Python SDK Notebooks

Azure Machine Learning Python SDK notebooks

a community-driven repository of examples using mlflow for tracking can be found at

Welcome to the Azure Machine Learning Python SDK notebooks repository!

Getting started

These notebooks are recommended for use in an Azure Machine Learning Compute Instance, where you can run them without any additional set up.

However, the notebooks can be run in any development environment with the correct azureml packages installed.

Install the azureml.core Python package:

pip install azureml-core

Install additional packages as needed:

pip install azureml-mlflow
pip install azureml-dataset-runtime
pip install azureml-automl-runtime
pip install azureml-pipeline
pip install azureml-pipeline-steps

We recommend starting with one of the quickstarts.


This repository is a push-only mirror. Pull requests are ignored.

Code of Conduct

This project has adopted the Microsoft Open Source Code of Conduct. Please see the code of conduct for details.


Download Details:

Author: Azure
Source Code: 
License: MIT license

#machinelearning #notebooks #azure #datascience 

MachineLearningNotebooks: Azure Machine Learning Python SDK Notebooks

Jupyterhub: Multi-user Server for Jupyter Notebooks


With JupyterHub you can create a multi-user Hub that spawns, manages, and proxies multiple instances of the single-user Jupyter notebook server.

Project Jupyter created JupyterHub to support many users. The Hub can offer notebook servers to a class of students, a corporate data science workgroup, a scientific research project, or a high-performance computing group.

Technical overview

Three main actors make up JupyterHub:

  • multi-user Hub (tornado process)
  • configurable http proxy (node-http-proxy)
  • multiple single-user Jupyter notebook servers (Python/Jupyter/tornado)

Basic principles for operation are:

  • Hub launches a proxy.
  • The Proxy forwards all requests to Hub by default.
  • Hub handles login and spawns single-user servers on demand.
  • Hub configures proxy to forward URL prefixes to the single-user notebook servers.

JupyterHub also provides a REST API for administration of the Hub and its users.


Check prerequisites

A Linux/Unix based system

Python 3.6 or greater


If you are using conda, the nodejs and npm dependencies will be installed for you by conda.

If you are using pip, install a recent version (at least 12.0) of nodejs/npm.

If using the default PAM Authenticator, a pluggable authentication module (PAM).

TLS certificate and key for HTTPS communication

Domain name

Install packages

Using conda

To install JupyterHub along with its dependencies including nodejs/npm:

conda install -c conda-forge jupyterhub

If you plan to run notebook servers locally, install JupyterLab or Jupyter notebook:

conda install jupyterlab
conda install notebook

Using pip

JupyterHub can be installed with pip, and the proxy with npm:

npm install -g configurable-http-proxy
python3 -m pip install jupyterhub

If you plan to run notebook servers locally, you will need to install JupyterLab or Jupyter notebook:

python3 -m pip install --upgrade jupyterlab
python3 -m pip install --upgrade notebook

Run the Hub server

To start the Hub server, run the command:


Visit http://localhost:8000 in your browser, and sign in with your system username and password.

Note: To allow multiple users to sign in to the server, you will need to run the jupyterhub command as a privileged user, such as root. The wiki describes how to run the server as a less privileged user, which requires more configuration of the system.


The Getting Started section of the documentation explains the common steps in setting up JupyterHub.

The JupyterHub tutorial provides an in-depth video and sample configurations of JupyterHub.

Create a configuration file

To generate a default config file with settings and descriptions:

jupyterhub --generate-config

Start the Hub

To start the Hub on a specific url and port with https:

jupyterhub --ip --port 443 --ssl-key my_ssl.key --ssl-cert my_ssl.cert


PAMAuthenticatorDefault, built-in authenticator
OAuthenticatorOAuth + JupyterHub Authenticator = OAuthenticator
ldapauthenticatorSimple LDAP Authenticator Plugin for JupyterHub
kerberosauthenticatorKerberos Authenticator Plugin for JupyterHub


LocalProcessSpawnerDefault, built-in spawner starts single-user servers as local processes
dockerspawnerSpawn single-user servers in Docker containers
kubespawnerKubernetes spawner for JupyterHub
sudospawnerSpawn single-user servers without being root
systemdspawnerSpawn single-user notebook servers using systemd
batchspawnerDesigned for clusters using batch scheduling software
yarnspawnerSpawn single-user notebook servers distributed on a Hadoop cluster
wrapspawnerWrapSpawner and ProfilesSpawner enabling runtime configuration of spawners


A starter docker image for JupyterHub gives a baseline deployment of JupyterHub using Docker.

Important: This jupyterhub/jupyterhub image contains only the Hub itself, with no configuration. In general, one needs to make a derivative image, with at least a setting up an Authenticator and/or a Spawner. To run the single-user servers, which may be on the same system as the Hub or not, Jupyter Notebook version 4 or greater must be installed.

The JupyterHub docker image can be started with the following command:

docker run -p 8000:8000 -d --name jupyterhub jupyterhub/jupyterhub jupyterhub

This command will create a container named jupyterhub that you can stop and resume with docker stop/start.

The Hub service will be listening on all interfaces at port 8000, which makes this a good choice for testing JupyterHub on your desktop or laptop.

If you want to run docker on a computer that has a public IP then you should (as in MUST) secure it with ssl by adding ssl options to your docker configuration or by using an ssl enabled proxy.

Mounting volumes will allow you to store data outside the docker image (host system) so it will be persistent, even when you start a new image.

The command docker exec -it jupyterhub bash will spawn a root shell in your docker container. You can use the root shell to create system users in the container. These accounts will be used for authentication in JupyterHub's default configuration.


If you would like to contribute to the project, please read our contributor documentation and the The file explains how to set up a development installation, how to run the test suite, and how to contribute to documentation.

For a high-level view of the vision and next directions of the project, see the JupyterHub community roadmap.

A note about platform support

JupyterHub is supported on Linux/Unix based systems.

JupyterHub officially does not support Windows. You may be able to use JupyterHub on Windows if you use a Spawner and Authenticator that work on Windows, but the JupyterHub defaults will not. Bugs reported on Windows will not be accepted, and the test suite will not run on Windows. Small patches that fix minor Windows compatibility issues (such as basic installation) may be accepted, however. For Windows-based systems, we would recommend running JupyterHub in a docker container or Linux VM.

Additional Reference: Tornado's documentation on Windows platform support

Help and resources

We encourage you to ask questions and share ideas on the Jupyter community forum. You can also talk with us on our JupyterHub Gitter channel.

JupyterHub follows the Jupyter Community Guides.

Technical Overview | Installation | Configuration | Docker | Contributing | License | Help and Resources

Please note that this repository is participating in a study into the sustainability of open source projects. Data will be gathered about this repository for approximately the next 12 months, starting from 2021-06-11.

Data collected will include the number of contributors, number of PRs, time taken to close/merge these PRs, and issues closed.

For more information, please visit our informational page or download our participant information sheet.

Download Details:

Author: jupyterhub
Source Code: 
License: View license

#jupyter #notebooks #python #jupyterhub 

Jupyterhub: Multi-user Server for Jupyter Notebooks
Monty  Boehm

Monty Boehm


BioTutorials: Tutorial Notebooks Of BioJulia

BioJulia Tutorial Notebooks

Pairwise Alignment and Dotplot (Binder)

# # Pairwise Alignment and Dotplot
# **Requirements**:
# - Julia 1.6+
# - BioAlignments v2
# - BioSequences v2
# Pairwise sequence alignment algorithms are provided by the `BioAlignments.jl` package.
# The sequence alignment is a kind of optimization problem to find the best scoring alignment.
# For example, matching two similar symbols gets a high (often positive) score
# and matching dissimilar symbols does a low (often negative) score.

using BioSequences
using BioAlignments

# The following two sequences are homologus protein sequences from [titin](
# of human and mouse.
# Titin is a component of muscle fiber and extremely long (~30,000 amino acids)
# compared to other protein sequences.
# So, the sequences used here are just part of them. 
# The complete sequences can be found here:

titin_human = aa"""


titin_mouse = aa"""

# The similarity score between two symbols are described by a [substitution matrix](
# and `BioAlignments.jl` provides some predefined matrices commonly seen in bioinformatics.
# `BLOSUM62` is a substitution matrix for amino acids
# and sometimes used as a default matrix in some programs.
# The derivation method and the biological rationale are outlined in this [wikipedia article](


# The score in the matrix can be accessed using a pair of amino acids:


# Two protein sequences are aligned to each other using the `pairalign` function,
# which takes an alignment optimization problem (`GlobalAlignment()`),
# two compared sequences (`titin_human`, `titin_mouse`) and scoring model (`model`).
# The algorithms implemented in this module can find the optimal sequence alignments
# between two sequences based on affine gap scoring model,
# in which inserting gaps of length `k` in a sequence alignment
# gets `gap_open + gap_extend * k` where `gap_open` and `gap_extend`
# are non-positive scores imposed on starting a new gap and extending a gap, respectively.

## Create a socring model from a substitution matrix and gap scores.
model = AffineGapScoreModel(BLOSUM62, gap_open=-11, gap_extend=-1)


## Solve a pairwise-alignment optimization problem.
result = pairalign(GlobalAlignment(), titin_human, titin_mouse, model)

# The achieved score of the alignment is then:


# The global alignment demonstrated above tries to align two sequences in an end-to-end manner.
# However, a researcher may be interested in local similarities between two sequences.
# In such a case, a [dot plot](
# is a good visualization to detect local similarities.

using CairoMakie

    dotplot(seq1, seq2, submat; windowsize=20, threshold=20)
Create a dot plot from `seq1` and `seq2` based on `submat`;
`windowsize` specifies the size of a window to accumulate similarity scores along two sequences and
`threshold` specifies the score threshold to plot a white dot in a window (when `threshold` is NaN,
it plots raw similarity scores).
function dotplot(seq1, seq2, submat; windowsize=20, threshold=20)
    m = length(seq1)
    n = length(seq2)
    simscores = zeros(m - windowsize + 1, n - windowsize + 1)
    for i in 1:m-windowsize+1, j in 1:n-windowsize+1
        for k in 1:windowsize
            simscores[i,j] += submat[seq1[i+k-1],seq2[j+k-1]]

    fig = Figure(; resolution=(800,800))
    if isnan(threshold)
        colorscale = :viridis
        ax = Axis(fig[1,1]; title="dotplot (windowsize=$windowsize, raw similarity score)")
        colorscale = :greys
        ax = Axis(fig[1,1]; title="dotplot (windowsize=$windowsize, threshold=$threshold)")
        for i in eachindex(simscores)
            simscores[i] = ifelse(simscores[i] > threshold, 1, 0)
    heatmap!(simscores, colorscale=colorscale)
    return fig

# We can see a raw similarity score plot by setting `threshold` to `NaN`.

dotplot(titin_human, titin_mouse, BLOSUM62, threshold=NaN)

# Filtering by the accumulated score highlights similar regions by white dots.
# The diagonal breaking lines (from the left-down to the right-up corner)
# correspond to the global similarity
# between the titin sequences of human and mouse as shown above,
# and other shorter lines around it
# corerspond to local similarities that are not captured in the global alignment.

dotplot(titin_human, titin_mouse, BLOSUM62)

RNA-Seq Coverage (outdated)

# RNA-Seq Coverage
# **Requirements**:
# - Julia 1.6+
# - BioSequences.jl v2
# - BioAlignments.jl v2
# - Xam.jl
# - ProgressMeters.jl
# In this tutorial we uses a public RNA-Seq data downloaded from the Sequence Read Archive (SRA):
# Paired-end FASTQ sequence files were extracted from the archive file
# and aligned to a reference genome using a couple of non-julia programs

# ... TODO

The programs under this repository are distributed in the public domain unless otherwise specified.

Download Details:

Author: BioJulia
Source Code: 

#julia #notebooks 

BioTutorials: Tutorial Notebooks Of BioJulia

How to Use Jupyter Notebook in The Terminal / Shell/ Bash with Nbterm

In this Coding Tutorial, We'll learn how to use Jupyter Notebook within our Terminal / Shell / Bash. Jupyter Notebook is usually available on the web browser but thanks to `nbterm` open sourced by David Brochart (sponsored by  QuantStack) you can Use Jupyter Notebook from the Terminal / Shell /Bash with a similar rich experience like the web UI.

#python #data-science #workflow #terminal #jupyter #notebooks 

How to Use Jupyter Notebook in The Terminal / Shell/ Bash with Nbterm

How to Use Imagededup To Detect Duplicate Images in Python with CNN

In this Python tutorial, We'll learn how to use imagededup, a Python library to find out duplicate images (using CNN - Convolutional Neural Networks). This tutorial doesn't require you to understand or know Neural Networks or Deep Learning. In just a few lines of Python Code, We'd have a Deep Learning based Image Deduping Script. For this demonstration, We'll try to identify Duplicate Pokemons.s

github -

#python #kaggle #notebooks #deep-learning 

How to Use Imagededup To Detect Duplicate Images in Python with CNN

How to Build Speech Recognition for any Language with Transformers

This Video Tutorial explains step-by-step guide of the Colab Notebook Hugging Face Notebook has put together to Fine-Tune XLSR-Wav2Vec2 for low-resource ASR with 🤗 Transformers. ASR - Automatic Speech Recognition. We demo with Hindi dataset from Mozilla Common Voice.

#python #colab #notebooks 

How to Build Speech Recognition for any Language with Transformers

How to Push Notifications From Jupyter Notebook After Code Execution

In this Jupyter Notebook Tutorial, We'll learn about `jupyternotify` a Python package that lets us send Push Notifications from Jupyter Notebook after Code Execution. This is extremely helpful in Machine Learning or Deep Learning Model training to get to know about the completion of the training process or the progress of the Model training process.

Github -

#python #jupyter #notebooks #data-science 

How to Push Notifications From Jupyter Notebook After Code Execution

How to Get Native Jupyter Notebook Experience on Visual Studio Code

In this Programming tutorial, You'll learn how to get Native Jupyter Notebook experience on Visual Studio Code Insiders. Native Notebook on VSCode supports all the languages that your current Jupyter Notebook supports.

How to get Native Jupyter Notebook on VSCode 
   Step 1. Install VS Code Insiders
   Step 2 If not working with Python, make sure to have a Jupyter kernelspec that corresponds to the language you would like to use installed on your machine.
   Step 3. Install the Jupyter Extension
   Step 4. Open or create a notebook file and start coding!

#vscode #notebooks 

How to Get Native Jupyter Notebook Experience on Visual Studio Code

How to Use R and Julia on VSCode Jupyter Notebook

Microsoft has refactored its VSCode Jupyter Extension to support all Jupyter Noteboook Kernels.That means, You can use languages other than Python on your VSCode Jupyter Notebook. Languages like R and Julia. In this Video you'll learn how to use R and Julia on VSCode Jupyter Notebook

#vscode #jupyter #notebooks #python #r #julia 

How to Use R and Julia on VSCode Jupyter Notebook
Emilie  Okumu

Emilie Okumu


How to Create interactive Visualizations Displayed on Jupyter Notebook

Use Bokeh to create interactive visualizations displayed on Jupyter Notebook

#notebooks #jupyter 

How to Create interactive Visualizations Displayed on Jupyter Notebook

Notebooks in VS Code

A deep dive into what Notebooks are and how to use them in VS Code.

#notebooks #vs code

Notebooks in VS Code
Sadie  Cassin

Sadie Cassin


A Lightweight Python Robot Simulator for JupyterLab and Notebooks


A lightweight Python robot simulator for JupyterLab, Notebooks, and other Python environments.


  1. A lightweight mobile robotics simulator
  2. Usable in the classroom, research, or exploration
  3. Explore wheeled robots with range, cameras, smell, and light sensors
  4. Operate quickly without a huge amount of resources
  5. Create reproducible experiments
  6. Designed for exposition, experimentation, and analysis
  7. Sensors designed for somewhat realistic problems (such as image recognition)
  8. Especially designed to work easily with Machine Learning and Artificial Intelligence systems


For the core operations, you will need to install just aitk.robots:

pip install aitk.robots

To use the Jupyter enhancements, you’ll also need the browser-based extensions. You can install those with:

jupyter labextension install @jupyter-widgets/jupyterlab-manager

If not in a conda environment, then you will also need to:

jupyter nbextension enable --py widgetsnbextension

For additional information, please see:

Download Details:

Author: ArtificialIntelligenceToolkit
The Demo/Documentation: View The Demo/Documentation
Download Link: Download The Source Code
Official Website:

#python #jupyterlab #notebooks #ai #artificial-intelligence

A Lightweight Python Robot Simulator for JupyterLab and Notebooks

Analyzing data hosted in Azure Cosmos DB with Notebooks

In this article, we will learn how to analyze data hosted in Azure Cosmos DB using Notebooks.


Azure Cosmos DB is a NoSQL data repository that supports a variety of API types to consume the data hosted in it. Jupyter Notebooks is one of the popular methods to access data for almost any type of data repositories. These notebooks support various types of SDKs, programming languages and frameworks. Python is one of the most popular programming languages and is used for a variety of purposes including data analysis with libraries like pandas and numpy. These libraries can be very effective in processing schemaless data structures that are typically stored in Azure Cosmos DB. Let’s learn how we can analyze data hosted in Cosmos DB using Notebooks in this article.


It is assumed that an Azure Cosmos DB account is already in place. In case not, then create a new account by opening the service and clicking on the New button. While creating the account, it provides an option to enable notebooks, which is Off by default. Ensure to switch “On” this option, as this will enable notebooks once the account is created. If you already have an account created, and this option was not turned on, on the console we would get an option to enable the notebooks. So, either way, ensure that the notebook feature is turned on. This feature is in preview as of the draft of this article, so it’s not yet recommended for use in a production environment till it goes generally available.

Once you have the account created with this feature turned on, when you open the account it would look as shown below. Navigate to the data explorer menu option, and under the Notebooks section, you would be able to see a gallery of notebooks as well as notebooks created by the end-user. In a new account, we would neither have any data nor any notebooks. We would create both in this exercise. Once we have an account that looks like this, our pre-requisite to start this exercise is met.

#azure #notebooks #azure cosmos db

Analyzing data hosted in Azure Cosmos DB with Notebooks
Imani  Gorczany

Imani Gorczany


TWC9: .NET 5.0 Preview 7, Git News, Windows Terminal 1.2, and more!


We are back with another episode of This Week on Channel 9! Join us as Christina, clad in a Ben Gibbord classic, catches you up on the latest developer news.

Please leave a comment or email us at

Create a Free Account (Azure)

#azure platform #azure services #notebooks #github

TWC9: .NET 5.0 Preview 7, Git News, Windows Terminal 1.2, and more!