Toni  Schmidt

Toni Schmidt

1614812280

Introducing Glommio, a Thread per Core Crate for Rust & Linux

When it comes to reducing cloud costs, optimizing bottlenecks in your code can only take you so far. It may be time to rethink your architecture. Perhaps you’re looking for a new architecture that takes into account the capabilities that modern hardware and software make available. One such architecture is called “thread-per-core”. Research recently demonstrated that a thread-per-core architecture can improve tail latencies of applications by up to 71%. That sounds fantastic, but the machine efficiency gains of thread-per-core can easily be negated by the loss of developer productivity when your application developers have to adjust to a completely new way of doing things and deal with a set of arcane challenges that are specific to this particular model.

Datadog is not immune to those problems. We run a variety of datastores at scale, which ingest metrics at a very high throughput. And we, too, were starting to see how some of our components’ existing architecture was beginning to show limitations. Metrics data, with its very high distribution in space, looks like a prime candidate for a thread-per-core architecture—but we were concerned about keeping the effort manageable.

This article will explore the thread-per-core model with its advantages and challenges, and introduce Glommio (you can also find it on crates.io), our solution to this problem. Glommio allows Rust developers to write thread-per-core applications in an easy and manageable way.

What is thread-per-core?

We know that thread-per-core can deliver significant efficiency gains. But what is it? In simple terms, any moderately complex application has many tasks that it needs to perform: it may need to read data from a database, feed that data through a machine learning model, and then pass that result along the pipeline. Some of those tasks are naturally sequential, but many can be done in parallel. And since modern hardware keeps increasing the number of cores available for applications, it is important to efficiently use them to achieve good performance numbers.

The simplest and most time-tested way of doing that is by employing threads: for each of its internal tasks, the application may use a different thread. If a thread has available work to do, it will do it; otherwise, it will go to sleep and allow the next one to run.

#rust

What is GEEK

Buddha Community

Introducing Glommio, a Thread per Core Crate for Rust & Linux

Pyringe: Debugger Capable Of Attaching to & Injecting Code Into Python

DISCLAIMER: This is not an official google project, this is just something I wrote while at Google.

Pyringe

What this is

Pyringe is a python debugger capable of attaching to running processes, inspecting their state and even of injecting python code into them while they're running. With pyringe, you can list threads, get tracebacks, inspect locals/globals/builtins of running functions, all without having to prepare your program for it.

What this is not

A "Google project". It's my internship project that got open-sourced. Sorry for the confusion.

What do I need?

Pyringe internally uses gdb to do a lot of its heavy lifting, so you will need a fairly recent build of gdb (version 7.4 onwards, and only if gdb was configured with --with-python). You will also need the symbols for whatever build of python you're running.
On Fedora, the package you're looking for is python-debuginfo, on Debian it's called python2.7-dbg (adjust according to version). Arch Linux users: see issue #5, Ubuntu users can only debug the python-dbg binary (see issue #19).
Having Colorama will get you output in boldface, but it's optional.

How do I get it?

Get it from the Github repo, PyPI, or via pip (pip install pyringe).

Is this Python3-friendly?

Short answer: No, sorry. Long answer:
There's three potentially different versions of python in play here:

  1. The version running pyringe
  2. The version being debugged
  3. The version of libpythonXX.so your build of gdb was linked against

2 Is currently the dealbreaker here. Cpython has changed a bit in the meantime[1], and making all features work while debugging python3 will have to take a back seat for now until the more glaring issues have been taken care of.
As for 1 and 3, the 2to3 tool may be able to handle it automatically. But then, as long as 2 hasn't been taken care of, this isn't really a use case in the first place.

[1] - For example, pendingbusy (which is used for injection) has been renamed to busy and been given a function-local scope, making it harder to interact with via gdb.

Will this work with PyPy?

Unfortunately, no. Since this makes use of some CPython internals and implementation details, only CPython is supported. If you don't know what PyPy or CPython are, you'll probably be fine.

Why not PDB?

PDB is great. Use it where applicable! But sometimes it isn't.
Like when python itself crashes, gets stuck in some C extension, or you want to inspect data without stopping a program. In such cases, PDB (and all other debuggers that run within the interpreter itself) are next to useless, and without pyringe you'd be left with having to debug using print statements. Pyringe is just quite convenient in these cases.

I injected a change to a local var into a function and it's not showing up!

This is a known limitation. Things like inject('var = 2') won't work, but inject('var[1] = 1337') should. This is because most of the time, python internally uses a fast path for looking up local variables that doesn't actually perform the dictionary lookup in locals(). In general, code you inject into processes with pyringe is very different from a normal python function call.

How do I use it?

You can start the debugger by executing python -m pyringe. Alternatively:

import pyringe
pyringe.interact()

If that reminds you of the code module, good; this is intentional.
After starting the debugger, you'll be greeted by what behaves almost like a regular python REPL.
Try the following:

==> pid:[None] #threads:[0] current thread:[None]
>>> help()
Available commands:
 attach: Attach to the process with the given pid.
 bt: Get a backtrace of the current position.
 [...]
==> pid:[None] #threads:[0] current thread:[None]
>>> attach(12679)
==> pid:[12679] #threads:[11] current thread:[140108099462912]
>>> threads()
[140108099462912, 140108107855616, 140108116248323, 140108124641024, 140108133033728, 140108224739072, 140108233131776, 140108141426432, 140108241524480, 140108249917184, 140108269324032]

The IDs you see here correspond to what threading.current_thread().ident would tell you.
All debugger functions are just regular python functions that have been exposed to the REPL, so you can do things like the following.

==> pid:[12679] #threads:[11] current thread:[140108099462912]
>>> for tid in threads():
...   if not tid % 10:
...     thread(tid)
...     bt()
... 
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 524, in __bootstrap
    self.__bootstrap_inner()
  File "/usr/lib/python2.7/threading.py", line 551, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 504, in run
    self.__target(*self.__args, **self.__kwargs)
  File "./test.py", line 46, in Idle
    Thread_2_Func(1)
  File "./test.py", line 40, in Wait
    time.sleep(n)
==> pid:[12679] #threads:[11] current thread:[140108241524480]
>>> 

You can access the inferior's locals and inspect them like so:

==> pid:[12679] #threads:[11] current thread:[140108241524480]
>>> inflocals()
{'a': <proxy of A object at remote 0x1d9b290>, 'LOL': 'success!', 'b': <proxy of B object at remote 0x1d988c0>, 'n': 1}
==> pid:[12679] #threads:[11] current thread:[140108241524480]
>>> p('a')
<proxy of A object at remote 0x1d9b290>
==> pid:[12679] #threads:[11] current thread:[140108241524480]
>>> p('a').attr
'Some_magic_string'
==> pid:[12679] #threads:[11] current thread:[140108241524480]
>>> 

And sure enough, the definition of a's class reads:

class Example(object):
  cl_attr = False
  def __init__(self):
    self.attr = 'Some_magic_string'

There's limits to how far this proxying of objects goes, and everything that isn't trivial data will show up as strings (like '<function at remote 0x1d957d0>').
You can inject python code into running programs. Of course, there are caveats but... see for yourself:

==> pid:[12679] #threads:[11] current thread:[140108241524480]
>>> inject('import threading')
==> pid:[12679] #threads:[11] current thread:[140108241524480]
>>> inject('print threading.current_thread().ident')
==> pid:[12679] #threads:[11] current thread:[140108241524480]
>>> 

The output of my program in this case reads:

140108241524480

If you need additional pointers, just try using python's help (pyhelp() in the debugger) on debugger commands.

Author: google
Source Code: https://github.com/google/pyringe
License: Apache-2.0 License

#python 

Einar  Hintz

Einar Hintz

1602560783

jQuery Ajax CRUD in ASP.NET Core MVC with Modal Popup

In this article, we’ll discuss how to use jQuery Ajax for ASP.NET Core MVC CRUD Operations using Bootstrap Modal. With jQuery Ajax, we can make HTTP request to controller action methods without reloading the entire page, like a single page application.

To demonstrate CRUD operations – insert, update, delete and retrieve, the project will be dealing with details of a normal bank transaction. GitHub repository for this demo project : https://bit.ly/33KTJAu.

Sub-topics discussed :

  • Form design for insert and update operation.
  • Display forms in modal popup dialog.
  • Form post using jQuery Ajax.
  • Implement MVC CRUD operations with jQuery Ajax.
  • Loading spinner in .NET Core MVC.
  • Prevent direct access to MVC action method.

Create ASP.NET Core MVC Project

In Visual Studio 2019, Go to File > New > Project (Ctrl + Shift + N).

From new project window, Select Asp.Net Core Web Application_._

Image showing how to create ASP.NET Core Web API project in Visual Studio.

Once you provide the project name and location. Select Web Application(Model-View-Controller) and uncheck HTTPS Configuration. Above steps will create a brand new ASP.NET Core MVC project.

Showing project template selection for .NET Core MVC.

Setup a Database

Let’s create a database for this application using Entity Framework Core. For that we’ve to install corresponding NuGet Packages. Right click on project from solution explorer, select Manage NuGet Packages_,_ From browse tab, install following 3 packages.

Showing list of NuGet Packages for Entity Framework Core

Now let’s define DB model class file – /Models/TransactionModel.cs.

public class TransactionModel
{
    [Key]
    public int TransactionId { get; set; }

    [Column(TypeName ="nvarchar(12)")]
    [DisplayName("Account Number")]
    [Required(ErrorMessage ="This Field is required.")]
    [MaxLength(12,ErrorMessage ="Maximum 12 characters only")]
    public string AccountNumber { get; set; }

    [Column(TypeName ="nvarchar(100)")]
    [DisplayName("Beneficiary Name")]
    [Required(ErrorMessage = "This Field is required.")]
    public string BeneficiaryName { get; set; }

    [Column(TypeName ="nvarchar(100)")]
    [DisplayName("Bank Name")]
    [Required(ErrorMessage = "This Field is required.")]
    public string BankName { get; set; }

    [Column(TypeName ="nvarchar(11)")]
    [DisplayName("SWIFT Code")]
    [Required(ErrorMessage = "This Field is required.")]
    [MaxLength(11)]
    public string SWIFTCode { get; set; }

    [DisplayName("Amount")]
    [Required(ErrorMessage = "This Field is required.")]
    public int Amount { get; set; }

    [DisplayFormat(DataFormatString = "{0:MM/dd/yyyy}")]
    public DateTime Date { get; set; }
}

C#Copy

Here we’ve defined model properties for the transaction with proper validation. Now let’s define  DbContextclass for EF Core.

#asp.net core article #asp.net core #add loading spinner in asp.net core #asp.net core crud without reloading #asp.net core jquery ajax form #asp.net core modal dialog #asp.net core mvc crud using jquery ajax #asp.net core mvc with jquery and ajax #asp.net core popup window #bootstrap modal popup in asp.net core mvc. bootstrap modal popup in asp.net core #delete and viewall in asp.net core #jquery ajax - insert #jquery ajax form post #modal popup dialog in asp.net core #no direct access action method #update #validation in modal popup

Toni  Schmidt

Toni Schmidt

1614812280

Introducing Glommio, a Thread per Core Crate for Rust & Linux

When it comes to reducing cloud costs, optimizing bottlenecks in your code can only take you so far. It may be time to rethink your architecture. Perhaps you’re looking for a new architecture that takes into account the capabilities that modern hardware and software make available. One such architecture is called “thread-per-core”. Research recently demonstrated that a thread-per-core architecture can improve tail latencies of applications by up to 71%. That sounds fantastic, but the machine efficiency gains of thread-per-core can easily be negated by the loss of developer productivity when your application developers have to adjust to a completely new way of doing things and deal with a set of arcane challenges that are specific to this particular model.

Datadog is not immune to those problems. We run a variety of datastores at scale, which ingest metrics at a very high throughput. And we, too, were starting to see how some of our components’ existing architecture was beginning to show limitations. Metrics data, with its very high distribution in space, looks like a prime candidate for a thread-per-core architecture—but we were concerned about keeping the effort manageable.

This article will explore the thread-per-core model with its advantages and challenges, and introduce Glommio (you can also find it on crates.io), our solution to this problem. Glommio allows Rust developers to write thread-per-core applications in an easy and manageable way.

What is thread-per-core?

We know that thread-per-core can deliver significant efficiency gains. But what is it? In simple terms, any moderately complex application has many tasks that it needs to perform: it may need to read data from a database, feed that data through a machine learning model, and then pass that result along the pipeline. Some of those tasks are naturally sequential, but many can be done in parallel. And since modern hardware keeps increasing the number of cores available for applications, it is important to efficiently use them to achieve good performance numbers.

The simplest and most time-tested way of doing that is by employing threads: for each of its internal tasks, the application may use a different thread. If a thread has available work to do, it will do it; otherwise, it will go to sleep and allow the next one to run.

#rust

Serde Rust: Serialization Framework for Rust

Serde

*Serde is a framework for serializing and deserializing Rust data structures efficiently and generically.*

You may be looking for:

Serde in action

Click to show Cargo.toml. Run this code in the playground.

[dependencies]

# The core APIs, including the Serialize and Deserialize traits. Always
# required when using Serde. The "derive" feature is only required when
# using #[derive(Serialize, Deserialize)] to make Serde work with structs
# and enums defined in your crate.
serde = { version = "1.0", features = ["derive"] }

# Each data format lives in its own crate; the sample code below uses JSON
# but you may be using a different one.
serde_json = "1.0"

 

use serde::{Serialize, Deserialize};

#[derive(Serialize, Deserialize, Debug)]
struct Point {
    x: i32,
    y: i32,
}

fn main() {
    let point = Point { x: 1, y: 2 };

    // Convert the Point to a JSON string.
    let serialized = serde_json::to_string(&point).unwrap();

    // Prints serialized = {"x":1,"y":2}
    println!("serialized = {}", serialized);

    // Convert the JSON string back to a Point.
    let deserialized: Point = serde_json::from_str(&serialized).unwrap();

    // Prints deserialized = Point { x: 1, y: 2 }
    println!("deserialized = {:?}", deserialized);
}

Getting help

Serde is one of the most widely used Rust libraries so any place that Rustaceans congregate will be able to help you out. For chat, consider trying the #rust-questions or #rust-beginners channels of the unofficial community Discord (invite: https://discord.gg/rust-lang-community), the #rust-usage or #beginners channels of the official Rust Project Discord (invite: https://discord.gg/rust-lang), or the #general stream in Zulip. For asynchronous, consider the [rust] tag on StackOverflow, the /r/rust subreddit which has a pinned weekly easy questions post, or the Rust Discourse forum. It's acceptable to file a support issue in this repo but they tend not to get as many eyes as any of the above and may get closed without a response after some time.

Download Details:
Author: serde-rs
Source Code: https://github.com/serde-rs/serde
License: View license

#rust  #rustlang 

Lydia  Kessler

Lydia Kessler

1626318000

ULTIMATE Rust Lang Tutorial! - Publishing a Rust Crate

The ultimate Rust lang tutorial. Follow along as we go through the Rust lang book chapter by chapter.

📝Get the FREE Rust Cheatsheet: https://letsgetrusty.com/cheatsheet

The Rust book: https://doc.rust-lang.org/stable/book/​​

Chapters:
0:00​ Intro
0:43 Release Profiles
3:00 Documentation Comments
4:32 Commonly Used Sections
5:04 Documentation Comments as Tests
5:50 Commenting Contained Items
6:29 Exporting a Public API
8:44 Setting up Creates.io Account
9:54 Adding Metadata to a New Create
12:14 Publishing to Crates.io
12:49 Removing Version from Crates.io
13:37 Outro

#letsgetrusty​​ #rust​lang​ #tutorial

#rust #rust lang #rust crate