Fabiola  Auma

Fabiola Auma

1675337640

How to Writing A Modern HTTP(S) Tunnel in Rust

Learn how to write performant and safe apps quickly in Rust. This post guides you through designing and implementing an HTTP Tunnel, and covers the basics of creating robust, scalable, and observable applications.

Rust: Performance, Reliability, Productivity

About a year ago, I started to learn Rust. The first two weeks were quite painful. Nothing compiled; I didn’t know how to do basic operations; I couldn’t make a simple program run. But step by step, I started to understand what the compiler wanted. Even more, I realized that it forces the right thinking and correct behavior.

Yes, sometimes, you have to write seemingly redundant constructs. But it’s better not to compile a correct program than to compile an incorrect one. This makes making mistakes more difficult.

Soon after, I became more or less productive and finally could do what I wanted. Well, most of the time.

Out of curiosity, I decided to take on a slightly more complex challenge: implement an HTTP Tunnel in Rust. It turned out to be surprisingly easy to do and took about a day, which is quite impressive. I stitched together tokio, clap, serde, and several other very useful crates. Let me share the knowledge I gained during this exciting challenge and elaborate on why I organized the app this way. I hope you’ll enjoy it.

What Is an HTTP Tunnel?

Simply put, it’s a lightweight VPN that you can set up with your browser so your internet provider cannot block or track your activity, and web servers won’t see your IP address.

If you’d like, you can test it with your browser locally, e.g., with Firefox, (otherwise, just skip this section for now).

Tutorial

1. Install the App Using Cargo

$ cargo install http-tunnel

2. Start

$ http-tunnel --bind 0.0.0.0:8080 http

 

You can also check the HTTP-tunnel GitHub repository for the build/installation instructions.

Now, you can go to your browser and set the HTTP Proxy to localhost:8080. For instance, in Firefox, search for proxy in the preferences section:

Proxy Search

Find the proxy settings, specify it for HTTP Proxy, and check it for HTTPS:

Connection Settings

Set the proxy to just built http_tunnel. You can visit several web pages and check the ./logs/application.log file—all your traffic was going via the tunnel. For example: 

Traffic

Now, let’s walk through the process from the beginning.

Design the App

Each application starts with a design, which means we need to define the following:

  • Functional requirements.
  • Non-functional requirements.
  • Application abstractions and components.

Functional Requirements

We need to follow the specification outlined in the here

Negotiate a target with an HTTP CONNECT request. For example, if the client wants to create a tunnel to Wikipedia’s website, the request will look like this:

CONNECT www.wikipedia.org:443 HTTP/1.1
...

 

Followed by a response like below:

HTTP/1.1 200 OK

 

After this point, just relay TCP traffic both ways until one of the sides closes it or an I/O error happens.

The HTTP Tunnel should work for both HTTP and HTTPS.

We also should be able to manage access/block targets (e.g., to block-list trackers).

Non-Functional Requirements

The service shouldn’t log any information that identifies users.

It should have high throughput and low latency (it should be unnoticeable for users and relatively cheap to run).

Ideally, we want it to be resilient to traffic spikes, provide noisy neighbor isolation, and resist basic DDoS attacks.

Error messaging should be developer-friendly. We want the system to be observable to troubleshoot and tune it in production at a massive scale.

Components

When designing components, we need to break down the app into a set of responsibilities. First, let’s see what our flow diagram looks like:

Flow Diagram

To implement this, we can introduce the following four main components:

  1. TCP/TLS acceptor
  2. HTTP CONNECT negotiator
  3. Target connector
  4. Full-duplex relay

Implementation

TCP/TLS acceptor

When we roughly know how to organize the app, it’s time to decide which dependencies we should use. For Rust, the best I/O library I know is tokio. In the tokio family, there are many libraries including tokio-tls, which makes things much simpler. So the TCP acceptor code would look like this:

  let mut tcp_listener = TcpListener::bind(&proxy_configuration.bind_address)
        .await
        .map_err(|e| {
            error!(
                "Error binding address {}: {}",
                &proxy_configuration.bind_address, e
            );
            e
        })?;

And then the whole acceptor loop + launching asynchronous connection handlers would be:

 loop {
        // Asynchronously wait for an inbound socket.
        let socket = tcp_listener.accept().await;

        let dns_resolver_ref = dns_resolver.clone();

        match socket {
            Ok((stream, _)) => {
                let config = config.clone();
                // handle accepted connections asynchronously
                tokio::spawn(async move { tunnel_stream(&config, stream, dns_resolver_ref).await });
            }
            Err(e) => error!("Failed TCP handshake {}", e),
        }
    }

Let’s break down what’s happening here. We accept a connection. If the operation was successful, use tokio::spawn to create a new task that will handle that connection. Memory/thread-safety management happens behind the scenes. Handling futures is hidden by the async/await syntax sugar.

However, there is one question. TcpStream and TlsStream are different objects, but handling both is precisely the same. Can we reuse the same code? In Rust, abstraction is achieved via Traits, which are super handy:

/// Tunnel via a client connection.
async fn tunnel_stream<C: AsyncRead + AsyncWrite + Send + Unpin + 'static>(
    config: &ProxyConfiguration,
    client_connection: C,
    dns_resolver: DnsResolver,
) -> io::Result<()> {...}

The stream must implement:

  • AsyncRead /Write: Allows us to read/write it asynchronously.
  • Send: To be able to send between threads.
  • Unpin: To be moveable (otherwise, we won’t be able to do async move and tokio::spawn to create an async task).
  • 'static : To denote that it may live until the application shutdown and doesn’t depend on any other object’s destruction.

Which our TCP/TLS streams exactly are. However, now we can see that it doesn’t have to be TCP/TLS streams. This code would work for UDP, QUIC, or ICMP. For example, it can wrap any protocol within any other protocol or itself.

In other words, this code is reusable, extendable, and ready for migration, which happens sooner or later.

HTTP connect negotiator and target connector 

Let’s pause for a second and think at a higher level. What if we can abstract from HTTP Tunnel, and need to implement a generic tunnel?

Generic Tunnel

  • We need to establish some transport-level connections (L4).
  • Negotiate a target (doesn’t really matter how: HTTP, PPv2, etc.).
  • Establish an L4 connection to the target.
  • Report success and start relaying data.

A target could be, for instance, another tunnel. Also, we can support different protocols. The core would stay the same.

We already saw that the tunnel_stream method already works with any L4 Client<->Tunnel connection.

#[async_trait]
pub trait TunnelTarget {
    type Addr;
    fn addr(&self) -> Self::Addr;
}

#[async_trait]
pub trait TargetConnector {
    type Target: TunnelTarget + Send + Sync + Sized;
    type Stream: AsyncRead + AsyncWrite + Send + Sized + 'static;

    async fn connect(&mut self, target: &Self::Target) -> io::Result<Self::Stream>;
}

Here, we specify two abstractions:

  1. TunnelTarget is just something that has an Addr — whatever it is.
  2. TargetConnector — can connect to that Addr and needs to return a stream that supports async I/O.

Okay, but what about the target negotiation? The tokio-utils crate already has an abstraction for that, named Framed streams (with corresponding Encoder/Decoder traits). We need to implement them for HTTP CONNECT (or any other proxy protocol). You can find the implementation here.

Relay

We only have one major component remaining — that relays data after the tunnel negotiation is done. tokio provides a method to split a stream into two halves: ReadHalf and WriteHalf. We can split client and target connections and relay them in both directions:

        let (client_recv, client_send) = io::split(client);
        let (target_recv, target_send) = io::split(target);

        let upstream_task =
            tokio::spawn(
                async move { 
                    upstream_relay.relay_data(client_recv, target_send).await
                });

        let downstream_task =
            tokio::spawn(
                async move { 
                    downstream_relay.relay_data(target_recv, client_send).await 
                });

Where the relay_data(…) definition requires nothing more than implementing the abstractions mentioned above. For example, it can connect any two halves of a stream:

/// Relays data in a single direction. E.g.
pub async fn relay_data<R: AsyncReadExt + Sized, W: AsyncWriteExt + Sized>(
        self,
        mut source: ReadHalf<R>,
        mut dest: WriteHalf<W>,
    ) -> io::Result<RelayStats> {...}

And finally, instead of a simple HTTP Tunnel, we have an engine that can be used to build any type of tunnels or a chain of tunnels (e.g., for onion routing) over any transport and proxy protocols:

/// A connection tunnel.
///
/// # Parameters
/// * `<H>` - proxy handshake codec for initiating a tunnel.
///    It extracts the request message, which contains the target, and, potentially policies.
///    It also takes care of encoding a response.
/// * `<C>` - a connection from from client.
/// * `<T>` - target connector. It takes result produced by the codec and establishes a connection
///           to a target.
///
/// Once the target connection is established, it relays data until any connection is closed or an
/// error happens.
impl<H, C, T> ConnectionTunnel<H, C, T>
where
    H: Decoder<Error = EstablishTunnelResult> + Encoder<EstablishTunnelResult>,
    H::Item: TunnelTarget + Sized + Display + Send + Sync,
    C: AsyncRead + AsyncWrite + Sized + Send + Unpin + 'static,
    T: TargetConnector<Target = H::Item>,
{...}

The implementation is almost trivial in basic cases, but we want our app to handle failures, and that’s the focus of the next section.

Dealing With Failures

The amount of time engineers deal with failures is proportional to the scale of a system. It’s easy to write happy-case code. Still, if it enters an irrecoverable state on the very first error, it’s painful to use. Besides that, your app will be used by other engineers, and there are very few things more irritating than cryptic/misleading error messages. If your code runs as a part of a large service, some people need to monitor and support it (e.g., SREs or DevOps), and it should be a pleasure for them to deal with your service.

What kind of failures may an HTTP Tunnel encounter?

It’s a good idea to enumerate all error codes that your app returns to the client. So it’s clear why a request failed if the operation can be tried again (or shouldn’t) if it’s an integration bug, or just network noise:

pub enum EstablishTunnelResult {
    /// Successfully connected to target.  
    Ok,
    /// Malformed request
    BadRequest,
    /// Target is not allowed
    Forbidden,
    /// Unsupported operation, however valid for the protocol.
    OperationNotAllowed,
    /// The client failed to send a tunnel request timely.
    RequestTimeout,
    /// Cannot connect to target.
    BadGateway,
    /// Connection attempt timed out.
    GatewayTimeout,
    /// Busy. Try again later.
    TooManyRequests,
    /// Any other error. E.g. an abrupt I/O error.
    ServerError,
}

Dealing with delays is crucial for a network app. If your operations don’t have timeouts, it’s a matter of time until all of your threads will be “Waiting for Godot,” or your app will exhaust all available resources and become unavailable. Here we delegate the timeout definition to RelayPolicy:

let read_result = self
      .relay_policy
      .timed_operation(source.read(&mut buffer))
      .await;

  if read_result.is_err() {
      shutdown_reason = RelayShutdownReasons::ReaderTimeout;
      break;
  }

  let n = match read_result.unwrap() {
      Ok(n) if n == 0 => {
          shutdown_reason = RelayShutdownReasons::GracefulShutdown;
          break;
      }
      Ok(n) => n,
      Err(e) => {
          error!(
              "{} failed to read. Err = {:?}, CTX={}",
              self.name, e, self.tunnel_ctx
          );
          shutdown_reason = RelayShutdownReasons::ReadError;
          break;
      }
  };

The relay policy can be configured like this:

relay_policy:
  idle_timeout: 10s
  min_rate_bpm: 1000
  max_rate_bps: 10000
  max_lifetime: 100s
  max_total_payload: 100mb

 

So we can limit activity per connection with max_rate_bps and detect idle clients with min_rate_bpm (so they don’t consume system resources than can be utilized more productively). A connection lifetime and total traffic may be bounded as well.

It goes without saying that each failure mode needs to be tested. It’s straightforward to do that in Rust, in general, and with tokio-test in particular:

   #[tokio::test]
    async fn test_timed_operation_timeout() {
        let time_duration = 1;
        let data = b"data on the wire";
        let mut mock_connection: Mock = Builder::new()
            .wait(Duration::from_secs(time_duration * 2))
            .read(data)
            .build();

        let relay_policy: RelayPolicy = RelayPolicyBuilder::default()
            .min_rate_bpm(1000)
            .max_rate_bps(100_000)
            .idle_timeout(Duration::from_secs(time_duration))
            .build()
            .unwrap();

        let mut buf = [0; 1024];
        let timed_future = relay_policy
            .timed_operation(mock_connection.read(&mut buf))
            .await;
        assert!(timed_future.is_err());
    }

The same goes for I/O errors:

#[tokio::test]
    async fn test_timed_operation_failed_io() {
        let mut mock_connection: Mock = Builder::new()
            .read_error(Error::from(ErrorKind::BrokenPipe))
            .build();

        let relay_policy: RelayPolicy = RelayPolicyBuilder::default()
            .min_rate_bpm(1000)
            .max_rate_bps(100_000)
            .idle_timeout(Duration::from_secs(5))
            .build()
            .unwrap();

        let mut buf = [0; 1024];
        let timed_future = relay_policy
            .timed_operation(mock_connection.read(&mut buf))
            .await;
        assert!(timed_future.is_ok()); // no timeout
        assert!(timed_future.unwrap().is_err()); // but io-error
    }

Logging and Metrics

I haven’t seen an application that failed only in ways anticipated by its developers. I’m not saying there are no such applications. Still, chances are, your app is going to encounter something you didn’t expect: data races, specific traffic patterns, dealing with traffic bursts, and legacy clients.

But, one of the most common types of failures is human failures, such as pushing bad code or configuration, which are inevitable in large projects. Anyway, we need to be able to deal with something we didn’t foresee. So we emit enough information that would allow us to detect failures and troubleshoot.

So we’d better log every error and important event with meaningful information and relevant context as well as statistics:

/// Stats after the relay is closed. Can be used for telemetry/monitoring.
#[derive(Builder, Clone, Debug, Serialize)]
pub struct RelayStats {
    pub shutdown_reason: RelayShutdownReasons,
    pub total_bytes: usize,
    pub event_count: usize,
    pub duration: Duration,
}

/// Statistics. No sensitive information.
#[derive(Serialize)]
pub struct TunnelStats {
    tunnel_ctx: TunnelCtx,
    result: EstablishTunnelResult,
    upstream_stats: Option<RelayStats>,
    downstream_stats: Option<RelayStats>,
}

Note: the tunnel_ctx: TunnelCtx field, which can be used to correlate metric records with log messages:

Configuration and Parameters

Last but not least, we’d like to be able to run our tunnel in different modes with different parameters. Here’s where serde and clap become handy:

#[derive(Parser, Debug)]
#[clap(author, version, about, long_about = None)]
#[clap(propagate_version = true)]
struct Cli {
    /// Configuration file.
    #[clap(long)]
    config: Option<String>,
    /// Bind address, e.g. 0.0.0.0:8443.
    #[clap(long)]
    bind: String,
    #[clap(subcommand)]
    command: Commands,
}

#[derive(Subcommand, Debug)]
enum Commands {
    Http(HttpOptions),
    Https(HttpsOptions),
    Tcp(TcpOptions),
}

In my opinion, clap makes dealing with command line parameters pleasant. Extraordinarily expressive and easy to maintain.

Configuration files can be easily handled with serde-yaml:

target_connection:
  dns_cache_ttl: 60s
  allowed_targets: "(?i)(wikipedia|rust-lang)\\.org:443$"
  connect_timeout: 10s
  relay_policy:
    idle_timeout: 10s
    min_rate_bpm: 1000
    max_rate_bps: 10000

 

Which corresponds to Rust structs:

#[derive(Deserialize, Clone)]
pub struct TargetConnectionConfig {
    #[serde(with = "humantime_serde")]
    pub dns_cache_ttl: Duration,
    #[serde(with = "serde_regex")]
    pub allowed_targets: Regex,
    #[serde(with = "humantime_serde")]
    pub connect_timeout: Duration,
    pub relay_policy: RelayPolicy,
}

#[derive(Builder, Deserialize, Clone)]
pub struct RelayPolicy {
    #[serde(with = "humantime_serde")]
    pub idle_timeout: Duration,
    /// Min bytes-per-minute (bpm)
    pub min_rate_bpm: u64,
    // Max bytes-per-second (bps)
    pub max_rate_bps: u64,
}

It doesn’t need any additional comments to make it readable and maintainable, which is beautiful.

Conclusion

As you can see from this quick overview, the Rust ecosystem already provides many building blocks, so you can focus on what you need to do rather than how. You didn’t see any memory/resources management or explicit thread safety (which often comes at the expense of concurrency) with impressive performance. Abstraction mechanisms are fantastic, so your code can be highly reusable. This task was a lot of fun, so I’ll try to take on the next challenge.

Original article sourced at: https://medium.com

#rust 

What is GEEK

Buddha Community

How to Writing A Modern HTTP(S) Tunnel in Rust

Ethan Macen

1675339514

Hello Guys, I just completed my divorce with my cheating husband of 10 years with two beautiful kids when i got a solid evidence of his unfaithfulness on extra marital affairs and his infidelity lifestyle, And the various applications he used to hide chats and lot of secret on his cell phone. Then, i decided to hire (Henryclarkethicalhacker@gmail com), an hacker and a PI just to be 100% sure because i don’t want to confront him or take any kinda step without proof, fact and figures because that could leads to defamation… It was with this great, honest, professional and trustworthy Man i got to know my husband is a professional cheater and has been using this app to hide most of his chats. This great hacker helped me broke to into his cell phone activities and wired everything he does on his cell phone directly to my phone and i was able to monitor and track him directly from my phone remotely without him knowing, whatsapp, text, call, contact Henry via on +17736092741, or  1201-430-5865.
 

Mike  Kozey

Mike Kozey

1656151740

Test_cov_console: Flutter Console Coverage Test

Flutter Console Coverage Test

This small dart tools is used to generate Flutter Coverage Test report to console

How to install

Add a line like this to your package's pubspec.yaml (and run an implicit flutter pub get):

dev_dependencies:
  test_cov_console: ^0.2.2

How to run

run the following command to make sure all flutter library is up-to-date

flutter pub get
Running "flutter pub get" in coverage...                            0.5s

run the following command to generate lcov.info on coverage directory

flutter test --coverage
00:02 +1: All tests passed!

run the tool to generate report from lcov.info

flutter pub run test_cov_console
---------------------------------------------|---------|---------|---------|-------------------|
File                                         |% Branch | % Funcs | % Lines | Uncovered Line #s |
---------------------------------------------|---------|---------|---------|-------------------|
lib/src/                                     |         |         |         |                   |
 print_cov.dart                              |  100.00 |  100.00 |   88.37 |...,149,205,206,207|
 print_cov_constants.dart                    |    0.00 |    0.00 |    0.00 |    no unit testing|
lib/                                         |         |         |         |                   |
 test_cov_console.dart                       |    0.00 |    0.00 |    0.00 |    no unit testing|
---------------------------------------------|---------|---------|---------|-------------------|
 All files with unit testing                 |  100.00 |  100.00 |   88.37 |                   |
---------------------------------------------|---------|---------|---------|-------------------|

Optional parameter

If not given a FILE, "coverage/lcov.info" will be used.
-f, --file=<FILE>                      The target lcov.info file to be reported
-e, --exclude=<STRING1,STRING2,...>    A list of contains string for files without unit testing
                                       to be excluded from report
-l, --line                             It will print Lines & Uncovered Lines only
                                       Branch & Functions coverage percentage will not be printed
-i, --ignore                           It will not print any file without unit testing
-m, --multi                            Report from multiple lcov.info files
-c, --csv                              Output to CSV file
-o, --output=<CSV-FILE>                Full path of output CSV file
                                       If not given, "coverage/test_cov_console.csv" will be used
-t, --total                            Print only the total coverage
                                       Note: it will ignore all other option (if any), except -m
-p, --pass=<MINIMUM>                   Print only the whether total coverage is passed MINIMUM value or not
                                       If the value >= MINIMUM, it will print PASSED, otherwise FAILED
                                       Note: it will ignore all other option (if any), except -m
-h, --help                             Show this help

example run the tool with parameters

flutter pub run test_cov_console --file=coverage/lcov.info --exclude=_constants,_mock
---------------------------------------------|---------|---------|---------|-------------------|
File                                         |% Branch | % Funcs | % Lines | Uncovered Line #s |
---------------------------------------------|---------|---------|---------|-------------------|
lib/src/                                     |         |         |         |                   |
 print_cov.dart                              |  100.00 |  100.00 |   88.37 |...,149,205,206,207|
lib/                                         |         |         |         |                   |
 test_cov_console.dart                       |    0.00 |    0.00 |    0.00 |    no unit testing|
---------------------------------------------|---------|---------|---------|-------------------|
 All files with unit testing                 |  100.00 |  100.00 |   88.37 |                   |
---------------------------------------------|---------|---------|---------|-------------------|

report for multiple lcov.info files (-m, --multi)

It support to run for multiple lcov.info files with the followings directory structures:
1. No root module
<root>/<module_a>
<root>/<module_a>/coverage/lcov.info
<root>/<module_a>/lib/src
<root>/<module_b>
<root>/<module_b>/coverage/lcov.info
<root>/<module_b>/lib/src
...
2. With root module
<root>/coverage/lcov.info
<root>/lib/src
<root>/<module_a>
<root>/<module_a>/coverage/lcov.info
<root>/<module_a>/lib/src
<root>/<module_b>
<root>/<module_b>/coverage/lcov.info
<root>/<module_b>/lib/src
...
You must run test_cov_console on <root> dir, and the report would be grouped by module, here is
the sample output for directory structure 'with root module':
flutter pub run test_cov_console --file=coverage/lcov.info --exclude=_constants,_mock --multi
---------------------------------------------|---------|---------|---------|-------------------|
File                                         |% Branch | % Funcs | % Lines | Uncovered Line #s |
---------------------------------------------|---------|---------|---------|-------------------|
lib/src/                                     |         |         |         |                   |
 print_cov.dart                              |  100.00 |  100.00 |   88.37 |...,149,205,206,207|
lib/                                         |         |         |         |                   |
 test_cov_console.dart                       |    0.00 |    0.00 |    0.00 |    no unit testing|
---------------------------------------------|---------|---------|---------|-------------------|
 All files with unit testing                 |  100.00 |  100.00 |   88.37 |                   |
---------------------------------------------|---------|---------|---------|-------------------|
---------------------------------------------|---------|---------|---------|-------------------|
File - module_a -                            |% Branch | % Funcs | % Lines | Uncovered Line #s |
---------------------------------------------|---------|---------|---------|-------------------|
lib/src/                                     |         |         |         |                   |
 print_cov.dart                              |  100.00 |  100.00 |   88.37 |...,149,205,206,207|
lib/                                         |         |         |         |                   |
 test_cov_console.dart                       |    0.00 |    0.00 |    0.00 |    no unit testing|
---------------------------------------------|---------|---------|---------|-------------------|
 All files with unit testing                 |  100.00 |  100.00 |   88.37 |                   |
---------------------------------------------|---------|---------|---------|-------------------|
---------------------------------------------|---------|---------|---------|-------------------|
File - module_b -                            |% Branch | % Funcs | % Lines | Uncovered Line #s |
---------------------------------------------|---------|---------|---------|-------------------|
lib/src/                                     |         |         |         |                   |
 print_cov.dart                              |  100.00 |  100.00 |   88.37 |...,149,205,206,207|
lib/                                         |         |         |         |                   |
 test_cov_console.dart                       |    0.00 |    0.00 |    0.00 |    no unit testing|
---------------------------------------------|---------|---------|---------|-------------------|
 All files with unit testing                 |  100.00 |  100.00 |   88.37 |                   |
---------------------------------------------|---------|---------|---------|-------------------|

Output to CSV file (-c, --csv, -o, --output)

flutter pub run test_cov_console -c --output=coverage/test_coverage.csv

#### sample CSV output file:
File,% Branch,% Funcs,% Lines,Uncovered Line #s
lib/,,,,
test_cov_console.dart,0.00,0.00,0.00,no unit testing
lib/src/,,,,
parser.dart,100.00,100.00,97.22,"97"
parser_constants.dart,100.00,100.00,100.00,""
print_cov.dart,100.00,100.00,82.91,"29,49,51,52,171,174,177,180,183,184,185,186,187,188,279,324,325,387,388,389,390,391,392,393,394,395,398"
print_cov_constants.dart,0.00,0.00,0.00,no unit testing
All files with unit testing,100.00,100.00,86.07,""

Installing

Use this package as an executable

Install it

You can install the package from the command line:

dart pub global activate test_cov_console

Use it

The package has the following executables:

$ test_cov_console

Use this package as a library

Depend on it

Run this command:

With Dart:

 $ dart pub add test_cov_console

With Flutter:

 $ flutter pub add test_cov_console

This will add a line like this to your package's pubspec.yaml (and run an implicit dart pub get):

dependencies:
  test_cov_console: ^0.2.2

Alternatively, your editor might support dart pub get or flutter pub get. Check the docs for your editor to learn more.

Import it

Now in your Dart code, you can use:

import 'package:test_cov_console/test_cov_console.dart';

example/lib/main.dart

import 'package:flutter/material.dart';

void main() {
  runApp(MyApp());
}

class MyApp extends StatelessWidget {
  // This widget is the root of your application.
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        // This is the theme of your application.
        //
        // Try running your application with "flutter run". You'll see the
        // application has a blue toolbar. Then, without quitting the app, try
        // changing the primarySwatch below to Colors.green and then invoke
        // "hot reload" (press "r" in the console where you ran "flutter run",
        // or simply save your changes to "hot reload" in a Flutter IDE).
        // Notice that the counter didn't reset back to zero; the application
        // is not restarted.
        primarySwatch: Colors.blue,
        // This makes the visual density adapt to the platform that you run
        // the app on. For desktop platforms, the controls will be smaller and
        // closer together (more dense) than on mobile platforms.
        visualDensity: VisualDensity.adaptivePlatformDensity,
      ),
      home: MyHomePage(title: 'Flutter Demo Home Page'),
    );
  }
}

class MyHomePage extends StatefulWidget {
  MyHomePage({Key? key, required this.title}) : super(key: key);

  // This widget is the home page of your application. It is stateful, meaning
  // that it has a State object (defined below) that contains fields that affect
  // how it looks.

  // This class is the configuration for the state. It holds the values (in this
  // case the title) provided by the parent (in this case the App widget) and
  // used by the build method of the State. Fields in a Widget subclass are
  // always marked "final".

  final String title;

  @override
  _MyHomePageState createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  int _counter = 0;

  void _incrementCounter() {
    setState(() {
      // This call to setState tells the Flutter framework that something has
      // changed in this State, which causes it to rerun the build method below
      // so that the display can reflect the updated values. If we changed
      // _counter without calling setState(), then the build method would not be
      // called again, and so nothing would appear to happen.
      _counter++;
    });
  }

  @override
  Widget build(BuildContext context) {
    // This method is rerun every time setState is called, for instance as done
    // by the _incrementCounter method above.
    //
    // The Flutter framework has been optimized to make rerunning build methods
    // fast, so that you can just rebuild anything that needs updating rather
    // than having to individually change instances of widgets.
    return Scaffold(
      appBar: AppBar(
        // Here we take the value from the MyHomePage object that was created by
        // the App.build method, and use it to set our appbar title.
        title: Text(widget.title),
      ),
      body: Center(
        // Center is a layout widget. It takes a single child and positions it
        // in the middle of the parent.
        child: Column(
          // Column is also a layout widget. It takes a list of children and
          // arranges them vertically. By default, it sizes itself to fit its
          // children horizontally, and tries to be as tall as its parent.
          //
          // Invoke "debug painting" (press "p" in the console, choose the
          // "Toggle Debug Paint" action from the Flutter Inspector in Android
          // Studio, or the "Toggle Debug Paint" command in Visual Studio Code)
          // to see the wireframe for each widget.
          //
          // Column has various properties to control how it sizes itself and
          // how it positions its children. Here we use mainAxisAlignment to
          // center the children vertically; the main axis here is the vertical
          // axis because Columns are vertical (the cross axis would be
          // horizontal).
          mainAxisAlignment: MainAxisAlignment.center,
          children: <Widget>[
            Text(
              'You have pushed the button this many times:',
            ),
            Text(
              '$_counter',
              style: Theme.of(context).textTheme.headline4,
            ),
          ],
        ),
      ),
      floatingActionButton: FloatingActionButton(
        onPressed: _incrementCounter,
        tooltip: 'Increment',
        child: Icon(Icons.add),
      ), // This trailing comma makes auto-formatting nicer for build methods.
    );
  }
}

Author: DigitalKatalis
Source Code: https://github.com/DigitalKatalis/test_cov_console 
License: BSD-3-Clause license

#flutter #dart #test 

Serde Rust: Serialization Framework for Rust

Serde

*Serde is a framework for serializing and deserializing Rust data structures efficiently and generically.*

You may be looking for:

Serde in action

Click to show Cargo.toml. Run this code in the playground.

[dependencies]

# The core APIs, including the Serialize and Deserialize traits. Always
# required when using Serde. The "derive" feature is only required when
# using #[derive(Serialize, Deserialize)] to make Serde work with structs
# and enums defined in your crate.
serde = { version = "1.0", features = ["derive"] }

# Each data format lives in its own crate; the sample code below uses JSON
# but you may be using a different one.
serde_json = "1.0"

 

use serde::{Serialize, Deserialize};

#[derive(Serialize, Deserialize, Debug)]
struct Point {
    x: i32,
    y: i32,
}

fn main() {
    let point = Point { x: 1, y: 2 };

    // Convert the Point to a JSON string.
    let serialized = serde_json::to_string(&point).unwrap();

    // Prints serialized = {"x":1,"y":2}
    println!("serialized = {}", serialized);

    // Convert the JSON string back to a Point.
    let deserialized: Point = serde_json::from_str(&serialized).unwrap();

    // Prints deserialized = Point { x: 1, y: 2 }
    println!("deserialized = {:?}", deserialized);
}

Getting help

Serde is one of the most widely used Rust libraries so any place that Rustaceans congregate will be able to help you out. For chat, consider trying the #rust-questions or #rust-beginners channels of the unofficial community Discord (invite: https://discord.gg/rust-lang-community), the #rust-usage or #beginners channels of the official Rust Project Discord (invite: https://discord.gg/rust-lang), or the #general stream in Zulip. For asynchronous, consider the [rust] tag on StackOverflow, the /r/rust subreddit which has a pinned weekly easy questions post, or the Rust Discourse forum. It's acceptable to file a support issue in this repo but they tend not to get as many eyes as any of the above and may get closed without a response after some time.

Download Details:
Author: serde-rs
Source Code: https://github.com/serde-rs/serde
License: View license

#rust  #rustlang 

Rust  Language

Rust Language

1636360749

Std Library Types in Rust - The Rust Programming Language

Std Library Types - Rust By Example

The std library provides many custom types which expands drastically on the primitives. Some of these include:

  • growable Strings like: "hello world"
  • growable vectors: [1, 2, 3]
  • optional types: Option<i32>
  • error handling types: Result<i32, i32>
  • heap allocated pointers: Box<i32>

Box, stack and heap

All values in Rust are stack allocated by default. Values can be boxed (allocated on the heap) by creating a Box<T>. A box is a smart pointer to a heap allocated value of type T. When a box goes out of scope, its destructor is called, the inner object is destroyed, and the memory on the heap is freed.

Boxed values can be dereferenced using the * operator; this removes one layer of indirection.

use std::mem;

#[allow(dead_code)]
#[derive(Debug, Clone, Copy)]
struct Point {
    x: f64,
    y: f64,
}

// A Rectangle can be specified by where its top left and bottom right 
// corners are in space
#[allow(dead_code)]
struct Rectangle {
    top_left: Point,
    bottom_right: Point,
}

fn origin() -> Point {
    Point { x: 0.0, y: 0.0 }
}

fn boxed_origin() -> Box<Point> {
    // Allocate this point on the heap, and return a pointer to it
    Box::new(Point { x: 0.0, y: 0.0 })
}

fn main() {
    // (all the type annotations are superfluous)
    // Stack allocated variables
    let point: Point = origin();
    let rectangle: Rectangle = Rectangle {
        top_left: origin(),
        bottom_right: Point { x: 3.0, y: -4.0 }
    };

    // Heap allocated rectangle
    let boxed_rectangle: Box<Rectangle> = Box::new(Rectangle {
        top_left: origin(),
        bottom_right: Point { x: 3.0, y: -4.0 },
    });

    // The output of functions can be boxed
    let boxed_point: Box<Point> = Box::new(origin());

    // Double indirection
    let box_in_a_box: Box<Box<Point>> = Box::new(boxed_origin());

    println!("Point occupies {} bytes on the stack",
             mem::size_of_val(&point));
    println!("Rectangle occupies {} bytes on the stack",
             mem::size_of_val(&rectangle));

    // box size == pointer size
    println!("Boxed point occupies {} bytes on the stack",
             mem::size_of_val(&boxed_point));
    println!("Boxed rectangle occupies {} bytes on the stack",
             mem::size_of_val(&boxed_rectangle));
    println!("Boxed box occupies {} bytes on the stack",
             mem::size_of_val(&box_in_a_box));

    // Copy the data contained in `boxed_point` into `unboxed_point`
    let unboxed_point: Point = *boxed_point;
    println!("Unboxed point occupies {} bytes on the stack",
             mem::size_of_val(&unboxed_point));
}

Vectors

Vectors are re-sizable arrays. Like slices, their size is not known at compile time, but they can grow or shrink at any time. A vector is represented using 3 parameters:

  • pointer to the data
  • length
  • capacity

The capacity indicates how much memory is reserved for the vector. The vector can grow as long as the length is smaller than the capacity. When this threshold needs to be surpassed, the vector is reallocated with a larger capacity.

fn main() {
    // Iterators can be collected into vectors
    let collected_iterator: Vec<i32> = (0..10).collect();
    println!("Collected (0..10) into: {:?}", collected_iterator);

    // The `vec!` macro can be used to initialize a vector
    let mut xs = vec![1i32, 2, 3];
    println!("Initial vector: {:?}", xs);

    // Insert new element at the end of the vector
    println!("Push 4 into the vector");
    xs.push(4);
    println!("Vector: {:?}", xs);

    // Error! Immutable vectors can't grow
    collected_iterator.push(0);
    // FIXME ^ Comment out this line

    // The `len` method yields the number of elements currently stored in a vector
    println!("Vector length: {}", xs.len());

    // Indexing is done using the square brackets (indexing starts at 0)
    println!("Second element: {}", xs[1]);

    // `pop` removes the last element from the vector and returns it
    println!("Pop last element: {:?}", xs.pop());

    // Out of bounds indexing yields a panic
    println!("Fourth element: {}", xs[3]);
    // FIXME ^ Comment out this line

    // `Vector`s can be easily iterated over
    println!("Contents of xs:");
    for x in xs.iter() {
        println!("> {}", x);
    }

    // A `Vector` can also be iterated over while the iteration
    // count is enumerated in a separate variable (`i`)
    for (i, x) in xs.iter().enumerate() {
        println!("In position {} we have value {}", i, x);
    }

    // Thanks to `iter_mut`, mutable `Vector`s can also be iterated
    // over in a way that allows modifying each value
    for x in xs.iter_mut() {
        *x *= 3;
    }
    println!("Updated vector: {:?}", xs);
}

More Vec methods can be found under the std::vec module


Strings

There are two types of strings in Rust: String and &str.

A String is stored as a vector of bytes (Vec<u8>), but guaranteed to always be a valid UTF-8 sequence. String is heap allocated, growable and not null terminated.

&str is a slice (&[u8]) that always points to a valid UTF-8 sequence, and can be used to view into a String, just like &[T] is a view into Vec<T>.

fn main() {
    // (all the type annotations are superfluous)
    // A reference to a string allocated in read only memory
    let pangram: &'static str = "the quick brown fox jumps over the lazy dog";
    println!("Pangram: {}", pangram);

    // Iterate over words in reverse, no new string is allocated
    println!("Words in reverse");
    for word in pangram.split_whitespace().rev() {
        println!("> {}", word);
    }

    // Copy chars into a vector, sort and remove duplicates
    let mut chars: Vec<char> = pangram.chars().collect();
    chars.sort();
    chars.dedup();

    // Create an empty and growable `String`
    let mut string = String::new();
    for c in chars {
        // Insert a char at the end of string
        string.push(c);
        // Insert a string at the end of string
        string.push_str(", ");
    }

    // The trimmed string is a slice to the original string, hence no new
    // allocation is performed
    let chars_to_trim: &[char] = &[' ', ','];
    let trimmed_str: &str = string.trim_matches(chars_to_trim);
    println!("Used characters: {}", trimmed_str);

    // Heap allocate a string
    let alice = String::from("I like dogs");
    // Allocate new memory and store the modified string there
    let bob: String = alice.replace("dog", "cat");

    println!("Alice says: {}", alice);
    println!("Bob says: {}", bob);
}

More str/String methods can be found under the std::str and std::string modules

Literals and escapes

There are multiple ways to write string literals with special characters in them. All result in a similar &str so it's best to use the form that is the most convenient to write. Similarly there are multiple ways to write byte string literals, which all result in &[u8; N].

Generally special characters are escaped with a backslash character: \. This way you can add any character to your string, even unprintable ones and ones that you don't know how to type. If you want a literal backslash, escape it with another one: \\

String or character literal delimiters occuring within a literal must be escaped: "\"", '\''.

fn main() {
    // You can use escapes to write bytes by their hexadecimal values...
    let byte_escape = "I'm writing \x52\x75\x73\x74!";
    println!("What are you doing\x3F (\\x3F means ?) {}", byte_escape);

    // ...or Unicode code points.
    let unicode_codepoint = "\u{211D}";
    let character_name = "\"DOUBLE-STRUCK CAPITAL R\"";

    println!("Unicode character {} (U+211D) is called {}",
                unicode_codepoint, character_name );


    let long_string = "String literals
                        can span multiple lines.
                        The linebreak and indentation here ->\
                        <- can be escaped too!";
    println!("{}", long_string);
}

Sometimes there are just too many characters that need to be escaped or it's just much more convenient to write a string out as-is. This is where raw string literals come into play.

fn main() {
    let raw_str = r"Escapes don't work here: \x3F \u{211D}";
    println!("{}", raw_str);

    // If you need quotes in a raw string, add a pair of #s
    let quotes = r#"And then I said: "There is no escape!""#;
    println!("{}", quotes);

    // If you need "# in your string, just use more #s in the delimiter.
    // There is no limit for the number of #s you can use.
    let longer_delimiter = r###"A string with "# in it. And even "##!"###;
    println!("{}", longer_delimiter);
}

Want a string that's not UTF-8? (Remember, str and String must be valid UTF-8). Or maybe you want an array of bytes that's mostly text? Byte strings to the rescue!

use std::str;

fn main() {
    // Note that this is not actually a `&str`
    let bytestring: &[u8; 21] = b"this is a byte string";

    // Byte arrays don't have the `Display` trait, so printing them is a bit limited
    println!("A byte string: {:?}", bytestring);

    // Byte strings can have byte escapes...
    let escaped = b"\x52\x75\x73\x74 as bytes";
    // ...but no unicode escapes
    // let escaped = b"\u{211D} is not allowed";
    println!("Some escaped bytes: {:?}", escaped);


    // Raw byte strings work just like raw strings
    let raw_bytestring = br"\u{211D} is not escaped here";
    println!("{:?}", raw_bytestring);

    // Converting a byte array to `str` can fail
    if let Ok(my_str) = str::from_utf8(raw_bytestring) {
        println!("And the same as text: '{}'", my_str);
    }

    let _quotes = br#"You can also use "fancier" formatting, \
                    like with normal raw strings"#;

    // Byte strings don't have to be UTF-8
    let shift_jis = b"\x82\xe6\x82\xa8\x82\xb1\x82\xbb"; // "ようこそ" in SHIFT-JIS

    // But then they can't always be converted to `str`
    match str::from_utf8(shift_jis) {
        Ok(my_str) => println!("Conversion successful: '{}'", my_str),
        Err(e) => println!("Conversion failed: {:?}", e),
    };
}

For conversions between character encodings check out the encoding crate.

A more detailed listing of the ways to write string literals and escape characters is given in the 'Tokens' chapter of the Rust Reference.


Option

Sometimes it's desirable to catch the failure of some parts of a program instead of calling panic!; this can be accomplished using the Option enum.

The Option<T> enum has two variants:

  • None, to indicate failure or lack of value, and
  • Some(value), a tuple struct that wraps a value with type T.
// An integer division that doesn't `panic!`
fn checked_division(dividend: i32, divisor: i32) -> Option<i32> {
    if divisor == 0 {
        // Failure is represented as the `None` variant
        None
    } else {
        // Result is wrapped in a `Some` variant
        Some(dividend / divisor)
    }
}

// This function handles a division that may not succeed
fn try_division(dividend: i32, divisor: i32) {
    // `Option` values can be pattern matched, just like other enums
    match checked_division(dividend, divisor) {
        None => println!("{} / {} failed!", dividend, divisor),
        Some(quotient) => {
            println!("{} / {} = {}", dividend, divisor, quotient)
        },
    }
}

fn main() {
    try_division(4, 2);
    try_division(1, 0);

    // Binding `None` to a variable needs to be type annotated
    let none: Option<i32> = None;
    let _equivalent_none = None::<i32>;

    let optional_float = Some(0f32);

    // Unwrapping a `Some` variant will extract the value wrapped.
    println!("{:?} unwraps to {:?}", optional_float, optional_float.unwrap());

    // Unwrapping a `None` variant will `panic!`
    println!("{:?} unwraps to {:?}", none, none.unwrap());
}

Result

We've seen that the Option enum can be used as a return value from functions that may fail, where None can be returned to indicate failure. However, sometimes it is important to express why an operation failed. To do this we have the Result enum.

The Result<T, E> enum has two variants:

  • Ok(value) which indicates that the operation succeeded, and wraps the value returned by the operation. (value has type T)
  • Err(why), which indicates that the operation failed, and wraps why, which (hopefully) explains the cause of the failure. (why has type E)
mod checked {
    // Mathematical "errors" we want to catch
    #[derive(Debug)]
    pub enum MathError {
        DivisionByZero,
        NonPositiveLogarithm,
        NegativeSquareRoot,
    }

    pub type MathResult = Result<f64, MathError>;

    pub fn div(x: f64, y: f64) -> MathResult {
        if y == 0.0 {
            // This operation would `fail`, instead let's return the reason of
            // the failure wrapped in `Err`
            Err(MathError::DivisionByZero)
        } else {
            // This operation is valid, return the result wrapped in `Ok`
            Ok(x / y)
        }
    }

    pub fn sqrt(x: f64) -> MathResult {
        if x < 0.0 {
            Err(MathError::NegativeSquareRoot)
        } else {
            Ok(x.sqrt())
        }
    }

    pub fn ln(x: f64) -> MathResult {
        if x <= 0.0 {
            Err(MathError::NonPositiveLogarithm)
        } else {
            Ok(x.ln())
        }
    }
}

// `op(x, y)` === `sqrt(ln(x / y))`
fn op(x: f64, y: f64) -> f64 {
    // This is a three level match pyramid!
    match checked::div(x, y) {
        Err(why) => panic!("{:?}", why),
        Ok(ratio) => match checked::ln(ratio) {
            Err(why) => panic!("{:?}", why),
            Ok(ln) => match checked::sqrt(ln) {
                Err(why) => panic!("{:?}", why),
                Ok(sqrt) => sqrt,
            },
        },
    }
}

fn main() {
    // Will this fail?
    println!("{}", op(1.0, 10.0));
}

?

Chaining results using match can get pretty untidy; luckily, the ? operator can be used to make things pretty again. ? is used at the end of an expression returning a Result, and is equivalent to a match expression, where the Err(err) branch expands to an early Err(From::from(err)), and the Ok(ok) branch expands to an ok expression.

mod checked {
    #[derive(Debug)]
    enum MathError {
        DivisionByZero,
        NonPositiveLogarithm,
        NegativeSquareRoot,
    }

    type MathResult = Result<f64, MathError>;

    fn div(x: f64, y: f64) -> MathResult {
        if y == 0.0 {
            Err(MathError::DivisionByZero)
        } else {
            Ok(x / y)
        }
    }

    fn sqrt(x: f64) -> MathResult {
        if x < 0.0 {
            Err(MathError::NegativeSquareRoot)
        } else {
            Ok(x.sqrt())
        }
    }

    fn ln(x: f64) -> MathResult {
        if x <= 0.0 {
            Err(MathError::NonPositiveLogarithm)
        } else {
            Ok(x.ln())
        }
    }

    // Intermediate function
    fn op_(x: f64, y: f64) -> MathResult {
        // if `div` "fails", then `DivisionByZero` will be `return`ed
        let ratio = div(x, y)?;

        // if `ln` "fails", then `NonPositiveLogarithm` will be `return`ed
        let ln = ln(ratio)?;

        sqrt(ln)
    }

    pub fn op(x: f64, y: f64) {
        match op_(x, y) {
            Err(why) => panic!("{}", match why {
                MathError::NonPositiveLogarithm
                    => "logarithm of non-positive number",
                MathError::DivisionByZero
                    => "division by zero",
                MathError::NegativeSquareRoot
                    => "square root of negative number",
            }),
            Ok(value) => println!("{}", value),
        }
    }
}

fn main() {
    checked::op(1.0, 10.0);
}

Be sure to check the documentation, as there are many methods to map/compose Result.


panic!

The panic! macro can be used to generate a panic and start unwinding its stack. While unwinding, the runtime will take care of freeing all the resources owned by the thread by calling the destructor of all its objects.

Since we are dealing with programs with only one thread, panic! will cause the program to report the panic message and exit.

// Re-implementation of integer division (/)
fn division(dividend: i32, divisor: i32) -> i32 {
    if divisor == 0 {
        // Division by zero triggers a panic
        panic!("division by zero");
    } else {
        dividend / divisor
    }
}

// The `main` task
fn main() {
    // Heap allocated integer
    let _x = Box::new(0i32);

    // This operation will trigger a task failure
    division(3, 0);

    println!("This point won't be reached!");

    // `_x` should get destroyed at this point
}

Let's check that panic! doesn't leak memory.

$ rustc panic.rs && valgrind ./panic
==4401== Memcheck, a memory error detector
==4401== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==4401== Using Valgrind-3.10.0.SVN and LibVEX; rerun with -h for copyright info
==4401== Command: ./panic
==4401== 
thread '<main>' panicked at 'division by zero', panic.rs:5
==4401== 
==4401== HEAP SUMMARY:
==4401==     in use at exit: 0 bytes in 0 blocks
==4401==   total heap usage: 18 allocs, 18 frees, 1,648 bytes allocated
==4401== 
==4401== All heap blocks were freed -- no leaks are possible
==4401== 
==4401== For counts of detected and suppressed errors, rerun with: -v
==4401== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

HashMap

Where vectors store values by an integer index, HashMaps store values by key. HashMap keys can be booleans, integers, strings, or any other type that implements the Eq and Hash traits. More on this in the next section.

Like vectors, HashMaps are growable, but HashMaps can also shrink themselves when they have excess space. You can create a HashMap with a certain starting capacity using HashMap::with_capacity(uint), or use HashMap::new() to get a HashMap with a default initial capacity (recommended).

use std::collections::HashMap;

fn call(number: &str) -> &str {
    match number {
        "798-1364" => "We're sorry, the call cannot be completed as dialed. 
            Please hang up and try again.",
        "645-7689" => "Hello, this is Mr. Awesome's Pizza. My name is Fred.
            What can I get for you today?",
        _ => "Hi! Who is this again?"
    }
}

fn main() { 
    let mut contacts = HashMap::new();

    contacts.insert("Daniel", "798-1364");
    contacts.insert("Ashley", "645-7689");
    contacts.insert("Katie", "435-8291");
    contacts.insert("Robert", "956-1745");

    // Takes a reference and returns Option<&V>
    match contacts.get(&"Daniel") {
        Some(&number) => println!("Calling Daniel: {}", call(number)),
        _ => println!("Don't have Daniel's number."),
    }

    // `HashMap::insert()` returns `None`
    // if the inserted value is new, `Some(value)` otherwise
    contacts.insert("Daniel", "164-6743");

    match contacts.get(&"Ashley") {
        Some(&number) => println!("Calling Ashley: {}", call(number)),
        _ => println!("Don't have Ashley's number."),
    }

    contacts.remove(&"Ashley"); 

    // `HashMap::iter()` returns an iterator that yields 
    // (&'a key, &'a value) pairs in arbitrary order.
    for (contact, &number) in contacts.iter() {
        println!("Calling {}: {}", contact, call(number)); 
    }
}

For more information on how hashing and hash maps (sometimes called hash tables) work, have a look at Hash Table Wikipedia

Alternate/custom key types

Any type that implements the Eq and Hash traits can be a key in HashMap. This includes:

  • bool (though not very useful since there is only two possible keys)
  • int, uint, and all variations thereof
  • String and &str (protip: you can have a HashMap keyed by String and call .get() with an &str)

Note that f32 and f64 do not implement Hash, likely because floating-point precision errors would make using them as hashmap keys horribly error-prone.

All collection classes implement Eq and Hash if their contained type also respectively implements Eq and Hash. For example, Vec<T> will implement Hash if T implements Hash.

You can easily implement Eq and Hash for a custom type with just one line: #[derive(PartialEq, Eq, Hash)]

The compiler will do the rest. If you want more control over the details, you can implement Eq and/or Hash yourself. This guide will not cover the specifics of implementing Hash.

To play around with using a struct in HashMap, let's try making a very simple user logon system:

use std::collections::HashMap;

// Eq requires that you derive PartialEq on the type.
#[derive(PartialEq, Eq, Hash)]
struct Account<'a>{
    username: &'a str,
    password: &'a str,
}

struct AccountInfo<'a>{
    name: &'a str,
    email: &'a str,
}

type Accounts<'a> = HashMap<Account<'a>, AccountInfo<'a>>;

fn try_logon<'a>(accounts: &Accounts<'a>,
        username: &'a str, password: &'a str){
    println!("Username: {}", username);
    println!("Password: {}", password);
    println!("Attempting logon...");

    let logon = Account {
        username,
        password,
    };

    match accounts.get(&logon) {
        Some(account_info) => {
            println!("Successful logon!");
            println!("Name: {}", account_info.name);
            println!("Email: {}", account_info.email);
        },
        _ => println!("Login failed!"),
    }
}

fn main(){
    let mut accounts: Accounts = HashMap::new();

    let account = Account {
        username: "j.everyman",
        password: "password123",
    };

    let account_info = AccountInfo {
        name: "John Everyman",
        email: "j.everyman@email.com",
    };

    accounts.insert(account, account_info);

    try_logon(&accounts, "j.everyman", "psasword123");

    try_logon(&accounts, "j.everyman", "password123");
}

HashSet

Consider a HashSet as a HashMap where we just care about the keys ( HashSet<T> is, in actuality, just a wrapper around HashMap<T, ()>).

"What's the point of that?" you ask. "I could just store the keys in a Vec."

A HashSet's unique feature is that it is guaranteed to not have duplicate elements. That's the contract that any set collection fulfills. HashSet is just one implementation. (see also: BTreeSet)

If you insert a value that is already present in the HashSet, (i.e. the new value is equal to the existing and they both have the same hash), then the new value will replace the old.

This is great for when you never want more than one of something, or when you want to know if you've already got something.

But sets can do more than that.

Sets have 4 primary operations (all of the following calls return an iterator):

union: get all the unique elements in both sets.

difference: get all the elements that are in the first set but not the second.

intersection: get all the elements that are only in both sets.

symmetric_difference: get all the elements that are in one set or the other, but not both.

Try all of these in the following example:

use std::collections::HashSet;

fn main() {
    let mut a: HashSet<i32> = vec![1i32, 2, 3].into_iter().collect();
    let mut b: HashSet<i32> = vec![2i32, 3, 4].into_iter().collect();

    assert!(a.insert(4));
    assert!(a.contains(&4));

    // `HashSet::insert()` returns false if
    // there was a value already present.
    assert!(b.insert(4), "Value 4 is already in set B!");
    // FIXME ^ Comment out this line

    b.insert(5);

    // If a collection's element type implements `Debug`,
    // then the collection implements `Debug`.
    // It usually prints its elements in the format `[elem1, elem2, ...]`
    println!("A: {:?}", a);
    println!("B: {:?}", b);

    // Print [1, 2, 3, 4, 5] in arbitrary order
    println!("Union: {:?}", a.union(&b).collect::<Vec<&i32>>());

    // This should print [1]
    println!("Difference: {:?}", a.difference(&b).collect::<Vec<&i32>>());

    // Print [2, 3, 4] in arbitrary order.
    println!("Intersection: {:?}", a.intersection(&b).collect::<Vec<&i32>>());

    // Print [1, 5]
    println!("Symmetric Difference: {:?}",
             a.symmetric_difference(&b).collect::<Vec<&i32>>());
}

(Examples are adapted from the documentation.)


Rc

When multiple ownership is needed, Rc(Reference Counting) can be used. Rc keeps track of the number of the references which means the number of owners of the value wrapped inside an Rc.

Reference count of an Rc increases by 1 whenever an Rc is cloned, and decreases by 1 whenever one cloned Rc is dropped out of the scope. When an Rc's reference count becomes zero, which means there are no owners remained, both the Rc and the value are all dropped.

Cloning an Rc never performs a deep copy. Cloning creates just another pointer to the wrapped value, and increments the count.

use std::rc::Rc;

fn main() {
    let rc_examples = "Rc examples".to_string();
    {
        println!("--- rc_a is created ---");
        
        let rc_a: Rc<String> = Rc::new(rc_examples);
        println!("Reference Count of rc_a: {}", Rc::strong_count(&rc_a));
        
        {
            println!("--- rc_a is cloned to rc_b ---");
            
            let rc_b: Rc<String> = Rc::clone(&rc_a);
            println!("Reference Count of rc_b: {}", Rc::strong_count(&rc_b));
            println!("Reference Count of rc_a: {}", Rc::strong_count(&rc_a));
            
            // Two `Rc`s are equal if their inner values are equal
            println!("rc_a and rc_b are equal: {}", rc_a.eq(&rc_b));
            
            // We can use methods of a value directly
            println!("Length of the value inside rc_a: {}", rc_a.len());
            println!("Value of rc_b: {}", rc_b);
            
            println!("--- rc_b is dropped out of scope ---");
        }
        
        println!("Reference Count of rc_a: {}", Rc::strong_count(&rc_a));
        
        println!("--- rc_a is dropped out of scope ---");
    }
    
    // Error! `rc_examples` already moved into `rc_a`
    // And when `rc_a` is dropped, `rc_examples` is dropped together
    // println!("rc_examples: {}", rc_examples);
    // TODO ^ Try uncommenting this line
}

Arc

When shared ownership between threads is needed, Arc(Atomic Reference Counted) can be used. This struct, via the Clone implementation can create a reference pointer for the location of a value in the memory heap while increasing the reference counter. As it shares ownership between threads, when the last reference pointer to a value is out of scope, the variable is dropped.


fn main() {
use std::sync::Arc;
use std::thread;

// This variable declaration is where its value is specified.
let apple = Arc::new("the same apple");

for _ in 0..10 {
    // Here there is no value specification as it is a pointer to a reference
    // in the memory heap.
    let apple = Arc::clone(&apple);

    thread::spawn(move || {
        // As Arc was used, threads can be spawned using the value allocated
        // in the Arc variable pointer's location.
        println!("{:?}", apple);
    });
}
}

Original article source at https://doc.rust-lang.org

#rust #programming #developer 

Rust  Language

Rust Language

1640144506

Strings - The Rust Programming Language

Rust For Beginners Tutorial - Strings

In this video we're taking a look at the String, &String and &str types in Rust!

Exercise solutions: https://github.com/PascalPrecht/rustlings/commits/solutions 

---
0:00 Intro
0:09 Exercise 1
4:47 Exercise 2
10:38 Outro


Strings

There are two types of strings in Rust: String and &str.

A String is stored as a vector of bytes (Vec<u8>), but guaranteed to always be a valid UTF-8 sequence. String is heap allocated, growable and not null terminated.

&str is a slice (&[u8]) that always points to a valid UTF-8 sequence, and can be used to view into a String, just like &[T] is a view into Vec<T>.

fn main() {
    // (all the type annotations are superfluous)
    // A reference to a string allocated in read only memory
    let pangram: &'static str = "the quick brown fox jumps over the lazy dog";
    println!("Pangram: {}", pangram);

    // Iterate over words in reverse, no new string is allocated
    println!("Words in reverse");
    for word in pangram.split_whitespace().rev() {
        println!("> {}", word);
    }

    // Copy chars into a vector, sort and remove duplicates
    let mut chars: Vec<char> = pangram.chars().collect();
    chars.sort();
    chars.dedup();

    // Create an empty and growable `String`
    let mut string = String::new();
    for c in chars {
        // Insert a char at the end of string
        string.push(c);
        // Insert a string at the end of string
        string.push_str(", ");
    }

    // The trimmed string is a slice to the original string, hence no new
    // allocation is performed
    let chars_to_trim: &[char] = &[' ', ','];
    let trimmed_str: &str = string.trim_matches(chars_to_trim);
    println!("Used characters: {}", trimmed_str);

    // Heap allocate a string
    let alice = String::from("I like dogs");
    // Allocate new memory and store the modified string there
    let bob: String = alice.replace("dog", "cat");

    println!("Alice says: {}", alice);
    println!("Bob says: {}", bob);
}

More str/String methods can be found under the std::str and std::string modules

Literals and escapes

There are multiple ways to write string literals with special characters in them. All result in a similar &str so it's best to use the form that is the most convenient to write. Similarly there are multiple ways to write byte string literals, which all result in &[u8; N].

Generally special characters are escaped with a backslash character: \. This way you can add any character to your string, even unprintable ones and ones that you don't know how to type. If you want a literal backslash, escape it with another one: \\

String or character literal delimiters occuring within a literal must be escaped: "\"", '\''.

fn main() {
    // You can use escapes to write bytes by their hexadecimal values...
    let byte_escape = "I'm writing \x52\x75\x73\x74!";
    println!("What are you doing\x3F (\\x3F means ?) {}", byte_escape);

    // ...or Unicode code points.
    let unicode_codepoint = "\u{211D}";
    let character_name = "\"DOUBLE-STRUCK CAPITAL R\"";

    println!("Unicode character {} (U+211D) is called {}",
                unicode_codepoint, character_name );


    let long_string = "String literals
                        can span multiple lines.
                        The linebreak and indentation here ->\
                        <- can be escaped too!";
    println!("{}", long_string);
}

Sometimes there are just too many characters that need to be escaped or it's just much more convenient to write a string out as-is. This is where raw string literals come into play.

fn main() {
    let raw_str = r"Escapes don't work here: \x3F \u{211D}";
    println!("{}", raw_str);

    // If you need quotes in a raw string, add a pair of #s
    let quotes = r#"And then I said: "There is no escape!""#;
    println!("{}", quotes);

    // If you need "# in your string, just use more #s in the delimiter.
    // There is no limit for the number of #s you can use.
    let longer_delimiter = r###"A string with "# in it. And even "##!"###;
    println!("{}", longer_delimiter);
}

Want a string that's not UTF-8? (Remember, str and String must be valid UTF-8). Or maybe you want an array of bytes that's mostly text? Byte strings to the rescue!

use std::str;

fn main() {
    // Note that this is not actually a `&str`
    let bytestring: &[u8; 21] = b"this is a byte string";

    // Byte arrays don't have the `Display` trait, so printing them is a bit limited
    println!("A byte string: {:?}", bytestring);

    // Byte strings can have byte escapes...
    let escaped = b"\x52\x75\x73\x74 as bytes";
    // ...but no unicode escapes
    // let escaped = b"\u{211D} is not allowed";
    println!("Some escaped bytes: {:?}", escaped);


    // Raw byte strings work just like raw strings
    let raw_bytestring = br"\u{211D} is not escaped here";
    println!("{:?}", raw_bytestring);

    // Converting a byte array to `str` can fail
    if let Ok(my_str) = str::from_utf8(raw_bytestring) {
        println!("And the same as text: '{}'", my_str);
    }

    let _quotes = br#"You can also use "fancier" formatting, \
                    like with normal raw strings"#;

    // Byte strings don't have to be UTF-8
    let shift_jis = b"\x82\xe6\x82\xa8\x82\xb1\x82\xbb"; // "ようこそ" in SHIFT-JIS

    // But then they can't always be converted to `str`
    match str::from_utf8(shift_jis) {
        Ok(my_str) => println!("Conversion successful: '{}'", my_str),
        Err(e) => println!("Conversion failed: {:?}", e),
    };
}

For conversions between character encodings check out the encoding crate.

A more detailed listing of the ways to write string literals and escape characters is given in the 'Tokens' chapter of the Rust Reference.

#rust #programming #developer 

Fabiola  Auma

Fabiola Auma

1675337640

How to Writing A Modern HTTP(S) Tunnel in Rust

Learn how to write performant and safe apps quickly in Rust. This post guides you through designing and implementing an HTTP Tunnel, and covers the basics of creating robust, scalable, and observable applications.

Rust: Performance, Reliability, Productivity

About a year ago, I started to learn Rust. The first two weeks were quite painful. Nothing compiled; I didn’t know how to do basic operations; I couldn’t make a simple program run. But step by step, I started to understand what the compiler wanted. Even more, I realized that it forces the right thinking and correct behavior.

Yes, sometimes, you have to write seemingly redundant constructs. But it’s better not to compile a correct program than to compile an incorrect one. This makes making mistakes more difficult.

Soon after, I became more or less productive and finally could do what I wanted. Well, most of the time.

Out of curiosity, I decided to take on a slightly more complex challenge: implement an HTTP Tunnel in Rust. It turned out to be surprisingly easy to do and took about a day, which is quite impressive. I stitched together tokio, clap, serde, and several other very useful crates. Let me share the knowledge I gained during this exciting challenge and elaborate on why I organized the app this way. I hope you’ll enjoy it.

What Is an HTTP Tunnel?

Simply put, it’s a lightweight VPN that you can set up with your browser so your internet provider cannot block or track your activity, and web servers won’t see your IP address.

If you’d like, you can test it with your browser locally, e.g., with Firefox, (otherwise, just skip this section for now).

Tutorial

1. Install the App Using Cargo

$ cargo install http-tunnel

2. Start

$ http-tunnel --bind 0.0.0.0:8080 http

 

You can also check the HTTP-tunnel GitHub repository for the build/installation instructions.

Now, you can go to your browser and set the HTTP Proxy to localhost:8080. For instance, in Firefox, search for proxy in the preferences section:

Proxy Search

Find the proxy settings, specify it for HTTP Proxy, and check it for HTTPS:

Connection Settings

Set the proxy to just built http_tunnel. You can visit several web pages and check the ./logs/application.log file—all your traffic was going via the tunnel. For example: 

Traffic

Now, let’s walk through the process from the beginning.

Design the App

Each application starts with a design, which means we need to define the following:

  • Functional requirements.
  • Non-functional requirements.
  • Application abstractions and components.

Functional Requirements

We need to follow the specification outlined in the here

Negotiate a target with an HTTP CONNECT request. For example, if the client wants to create a tunnel to Wikipedia’s website, the request will look like this:

CONNECT www.wikipedia.org:443 HTTP/1.1
...

 

Followed by a response like below:

HTTP/1.1 200 OK

 

After this point, just relay TCP traffic both ways until one of the sides closes it or an I/O error happens.

The HTTP Tunnel should work for both HTTP and HTTPS.

We also should be able to manage access/block targets (e.g., to block-list trackers).

Non-Functional Requirements

The service shouldn’t log any information that identifies users.

It should have high throughput and low latency (it should be unnoticeable for users and relatively cheap to run).

Ideally, we want it to be resilient to traffic spikes, provide noisy neighbor isolation, and resist basic DDoS attacks.

Error messaging should be developer-friendly. We want the system to be observable to troubleshoot and tune it in production at a massive scale.

Components

When designing components, we need to break down the app into a set of responsibilities. First, let’s see what our flow diagram looks like:

Flow Diagram

To implement this, we can introduce the following four main components:

  1. TCP/TLS acceptor
  2. HTTP CONNECT negotiator
  3. Target connector
  4. Full-duplex relay

Implementation

TCP/TLS acceptor

When we roughly know how to organize the app, it’s time to decide which dependencies we should use. For Rust, the best I/O library I know is tokio. In the tokio family, there are many libraries including tokio-tls, which makes things much simpler. So the TCP acceptor code would look like this:

  let mut tcp_listener = TcpListener::bind(&proxy_configuration.bind_address)
        .await
        .map_err(|e| {
            error!(
                "Error binding address {}: {}",
                &proxy_configuration.bind_address, e
            );
            e
        })?;

And then the whole acceptor loop + launching asynchronous connection handlers would be:

 loop {
        // Asynchronously wait for an inbound socket.
        let socket = tcp_listener.accept().await;

        let dns_resolver_ref = dns_resolver.clone();

        match socket {
            Ok((stream, _)) => {
                let config = config.clone();
                // handle accepted connections asynchronously
                tokio::spawn(async move { tunnel_stream(&config, stream, dns_resolver_ref).await });
            }
            Err(e) => error!("Failed TCP handshake {}", e),
        }
    }

Let’s break down what’s happening here. We accept a connection. If the operation was successful, use tokio::spawn to create a new task that will handle that connection. Memory/thread-safety management happens behind the scenes. Handling futures is hidden by the async/await syntax sugar.

However, there is one question. TcpStream and TlsStream are different objects, but handling both is precisely the same. Can we reuse the same code? In Rust, abstraction is achieved via Traits, which are super handy:

/// Tunnel via a client connection.
async fn tunnel_stream<C: AsyncRead + AsyncWrite + Send + Unpin + 'static>(
    config: &ProxyConfiguration,
    client_connection: C,
    dns_resolver: DnsResolver,
) -> io::Result<()> {...}

The stream must implement:

  • AsyncRead /Write: Allows us to read/write it asynchronously.
  • Send: To be able to send between threads.
  • Unpin: To be moveable (otherwise, we won’t be able to do async move and tokio::spawn to create an async task).
  • 'static : To denote that it may live until the application shutdown and doesn’t depend on any other object’s destruction.

Which our TCP/TLS streams exactly are. However, now we can see that it doesn’t have to be TCP/TLS streams. This code would work for UDP, QUIC, or ICMP. For example, it can wrap any protocol within any other protocol or itself.

In other words, this code is reusable, extendable, and ready for migration, which happens sooner or later.

HTTP connect negotiator and target connector 

Let’s pause for a second and think at a higher level. What if we can abstract from HTTP Tunnel, and need to implement a generic tunnel?

Generic Tunnel

  • We need to establish some transport-level connections (L4).
  • Negotiate a target (doesn’t really matter how: HTTP, PPv2, etc.).
  • Establish an L4 connection to the target.
  • Report success and start relaying data.

A target could be, for instance, another tunnel. Also, we can support different protocols. The core would stay the same.

We already saw that the tunnel_stream method already works with any L4 Client<->Tunnel connection.

#[async_trait]
pub trait TunnelTarget {
    type Addr;
    fn addr(&self) -> Self::Addr;
}

#[async_trait]
pub trait TargetConnector {
    type Target: TunnelTarget + Send + Sync + Sized;
    type Stream: AsyncRead + AsyncWrite + Send + Sized + 'static;

    async fn connect(&mut self, target: &Self::Target) -> io::Result<Self::Stream>;
}

Here, we specify two abstractions:

  1. TunnelTarget is just something that has an Addr — whatever it is.
  2. TargetConnector — can connect to that Addr and needs to return a stream that supports async I/O.

Okay, but what about the target negotiation? The tokio-utils crate already has an abstraction for that, named Framed streams (with corresponding Encoder/Decoder traits). We need to implement them for HTTP CONNECT (or any other proxy protocol). You can find the implementation here.

Relay

We only have one major component remaining — that relays data after the tunnel negotiation is done. tokio provides a method to split a stream into two halves: ReadHalf and WriteHalf. We can split client and target connections and relay them in both directions:

        let (client_recv, client_send) = io::split(client);
        let (target_recv, target_send) = io::split(target);

        let upstream_task =
            tokio::spawn(
                async move { 
                    upstream_relay.relay_data(client_recv, target_send).await
                });

        let downstream_task =
            tokio::spawn(
                async move { 
                    downstream_relay.relay_data(target_recv, client_send).await 
                });

Where the relay_data(…) definition requires nothing more than implementing the abstractions mentioned above. For example, it can connect any two halves of a stream:

/// Relays data in a single direction. E.g.
pub async fn relay_data<R: AsyncReadExt + Sized, W: AsyncWriteExt + Sized>(
        self,
        mut source: ReadHalf<R>,
        mut dest: WriteHalf<W>,
    ) -> io::Result<RelayStats> {...}

And finally, instead of a simple HTTP Tunnel, we have an engine that can be used to build any type of tunnels or a chain of tunnels (e.g., for onion routing) over any transport and proxy protocols:

/// A connection tunnel.
///
/// # Parameters
/// * `<H>` - proxy handshake codec for initiating a tunnel.
///    It extracts the request message, which contains the target, and, potentially policies.
///    It also takes care of encoding a response.
/// * `<C>` - a connection from from client.
/// * `<T>` - target connector. It takes result produced by the codec and establishes a connection
///           to a target.
///
/// Once the target connection is established, it relays data until any connection is closed or an
/// error happens.
impl<H, C, T> ConnectionTunnel<H, C, T>
where
    H: Decoder<Error = EstablishTunnelResult> + Encoder<EstablishTunnelResult>,
    H::Item: TunnelTarget + Sized + Display + Send + Sync,
    C: AsyncRead + AsyncWrite + Sized + Send + Unpin + 'static,
    T: TargetConnector<Target = H::Item>,
{...}

The implementation is almost trivial in basic cases, but we want our app to handle failures, and that’s the focus of the next section.

Dealing With Failures

The amount of time engineers deal with failures is proportional to the scale of a system. It’s easy to write happy-case code. Still, if it enters an irrecoverable state on the very first error, it’s painful to use. Besides that, your app will be used by other engineers, and there are very few things more irritating than cryptic/misleading error messages. If your code runs as a part of a large service, some people need to monitor and support it (e.g., SREs or DevOps), and it should be a pleasure for them to deal with your service.

What kind of failures may an HTTP Tunnel encounter?

It’s a good idea to enumerate all error codes that your app returns to the client. So it’s clear why a request failed if the operation can be tried again (or shouldn’t) if it’s an integration bug, or just network noise:

pub enum EstablishTunnelResult {
    /// Successfully connected to target.  
    Ok,
    /// Malformed request
    BadRequest,
    /// Target is not allowed
    Forbidden,
    /// Unsupported operation, however valid for the protocol.
    OperationNotAllowed,
    /// The client failed to send a tunnel request timely.
    RequestTimeout,
    /// Cannot connect to target.
    BadGateway,
    /// Connection attempt timed out.
    GatewayTimeout,
    /// Busy. Try again later.
    TooManyRequests,
    /// Any other error. E.g. an abrupt I/O error.
    ServerError,
}

Dealing with delays is crucial for a network app. If your operations don’t have timeouts, it’s a matter of time until all of your threads will be “Waiting for Godot,” or your app will exhaust all available resources and become unavailable. Here we delegate the timeout definition to RelayPolicy:

let read_result = self
      .relay_policy
      .timed_operation(source.read(&mut buffer))
      .await;

  if read_result.is_err() {
      shutdown_reason = RelayShutdownReasons::ReaderTimeout;
      break;
  }

  let n = match read_result.unwrap() {
      Ok(n) if n == 0 => {
          shutdown_reason = RelayShutdownReasons::GracefulShutdown;
          break;
      }
      Ok(n) => n,
      Err(e) => {
          error!(
              "{} failed to read. Err = {:?}, CTX={}",
              self.name, e, self.tunnel_ctx
          );
          shutdown_reason = RelayShutdownReasons::ReadError;
          break;
      }
  };

The relay policy can be configured like this:

relay_policy:
  idle_timeout: 10s
  min_rate_bpm: 1000
  max_rate_bps: 10000
  max_lifetime: 100s
  max_total_payload: 100mb

 

So we can limit activity per connection with max_rate_bps and detect idle clients with min_rate_bpm (so they don’t consume system resources than can be utilized more productively). A connection lifetime and total traffic may be bounded as well.

It goes without saying that each failure mode needs to be tested. It’s straightforward to do that in Rust, in general, and with tokio-test in particular:

   #[tokio::test]
    async fn test_timed_operation_timeout() {
        let time_duration = 1;
        let data = b"data on the wire";
        let mut mock_connection: Mock = Builder::new()
            .wait(Duration::from_secs(time_duration * 2))
            .read(data)
            .build();

        let relay_policy: RelayPolicy = RelayPolicyBuilder::default()
            .min_rate_bpm(1000)
            .max_rate_bps(100_000)
            .idle_timeout(Duration::from_secs(time_duration))
            .build()
            .unwrap();

        let mut buf = [0; 1024];
        let timed_future = relay_policy
            .timed_operation(mock_connection.read(&mut buf))
            .await;
        assert!(timed_future.is_err());
    }

The same goes for I/O errors:

#[tokio::test]
    async fn test_timed_operation_failed_io() {
        let mut mock_connection: Mock = Builder::new()
            .read_error(Error::from(ErrorKind::BrokenPipe))
            .build();

        let relay_policy: RelayPolicy = RelayPolicyBuilder::default()
            .min_rate_bpm(1000)
            .max_rate_bps(100_000)
            .idle_timeout(Duration::from_secs(5))
            .build()
            .unwrap();

        let mut buf = [0; 1024];
        let timed_future = relay_policy
            .timed_operation(mock_connection.read(&mut buf))
            .await;
        assert!(timed_future.is_ok()); // no timeout
        assert!(timed_future.unwrap().is_err()); // but io-error
    }

Logging and Metrics

I haven’t seen an application that failed only in ways anticipated by its developers. I’m not saying there are no such applications. Still, chances are, your app is going to encounter something you didn’t expect: data races, specific traffic patterns, dealing with traffic bursts, and legacy clients.

But, one of the most common types of failures is human failures, such as pushing bad code or configuration, which are inevitable in large projects. Anyway, we need to be able to deal with something we didn’t foresee. So we emit enough information that would allow us to detect failures and troubleshoot.

So we’d better log every error and important event with meaningful information and relevant context as well as statistics:

/// Stats after the relay is closed. Can be used for telemetry/monitoring.
#[derive(Builder, Clone, Debug, Serialize)]
pub struct RelayStats {
    pub shutdown_reason: RelayShutdownReasons,
    pub total_bytes: usize,
    pub event_count: usize,
    pub duration: Duration,
}

/// Statistics. No sensitive information.
#[derive(Serialize)]
pub struct TunnelStats {
    tunnel_ctx: TunnelCtx,
    result: EstablishTunnelResult,
    upstream_stats: Option<RelayStats>,
    downstream_stats: Option<RelayStats>,
}

Note: the tunnel_ctx: TunnelCtx field, which can be used to correlate metric records with log messages:

Configuration and Parameters

Last but not least, we’d like to be able to run our tunnel in different modes with different parameters. Here’s where serde and clap become handy:

#[derive(Parser, Debug)]
#[clap(author, version, about, long_about = None)]
#[clap(propagate_version = true)]
struct Cli {
    /// Configuration file.
    #[clap(long)]
    config: Option<String>,
    /// Bind address, e.g. 0.0.0.0:8443.
    #[clap(long)]
    bind: String,
    #[clap(subcommand)]
    command: Commands,
}

#[derive(Subcommand, Debug)]
enum Commands {
    Http(HttpOptions),
    Https(HttpsOptions),
    Tcp(TcpOptions),
}

In my opinion, clap makes dealing with command line parameters pleasant. Extraordinarily expressive and easy to maintain.

Configuration files can be easily handled with serde-yaml:

target_connection:
  dns_cache_ttl: 60s
  allowed_targets: "(?i)(wikipedia|rust-lang)\\.org:443$"
  connect_timeout: 10s
  relay_policy:
    idle_timeout: 10s
    min_rate_bpm: 1000
    max_rate_bps: 10000

 

Which corresponds to Rust structs:

#[derive(Deserialize, Clone)]
pub struct TargetConnectionConfig {
    #[serde(with = "humantime_serde")]
    pub dns_cache_ttl: Duration,
    #[serde(with = "serde_regex")]
    pub allowed_targets: Regex,
    #[serde(with = "humantime_serde")]
    pub connect_timeout: Duration,
    pub relay_policy: RelayPolicy,
}

#[derive(Builder, Deserialize, Clone)]
pub struct RelayPolicy {
    #[serde(with = "humantime_serde")]
    pub idle_timeout: Duration,
    /// Min bytes-per-minute (bpm)
    pub min_rate_bpm: u64,
    // Max bytes-per-second (bps)
    pub max_rate_bps: u64,
}

It doesn’t need any additional comments to make it readable and maintainable, which is beautiful.

Conclusion

As you can see from this quick overview, the Rust ecosystem already provides many building blocks, so you can focus on what you need to do rather than how. You didn’t see any memory/resources management or explicit thread safety (which often comes at the expense of concurrency) with impressive performance. Abstraction mechanisms are fantastic, so your code can be highly reusable. This task was a lot of fun, so I’ll try to take on the next challenge.

Original article sourced at: https://medium.com

#rust