1597441980
TeamSpeak 3 Server Query Library supports SSH and RAW Query
TS3-NodeJS-Library
The TS3 NodeJS Library has been strongly influenced by PlanetTeamSpeaks TS3 PHP Framework
Introduction
This library can connect to a TeamSpeak Server Query interface, send and receive commands aswell as upload and download files via Filetransfer!
With vscode you will receive powerful autocomplete of functions and their return values, so vscode is highly recommended!
Install
npm install --save ts3-nodejs-library
Documentation
You can find all necessary documentation here!
Example
Send a message to all non Query Clients connected:
//import with typescript
import { TeamSpeak, QueryProtocol } from "ts3-nodejs-library"
//import with javascript
//const { TeamSpeak } = require("ts3-nodejs-library")
//create a new connection
TeamSpeak.connect({
host: "localhost",
protocol: QueryProtocol.RAW, //optional
queryport: 10011, //optional
serverport: 9987,
username: "serveradmin",
password: "",
nickname: "NodeJS Query Framework"
}).then(async teamspeak => {
const clients = await teamspeak.clientList({ client_type: 0 })
clients.forEach(client => {
console.log("Sending 'Hello!' Message to", client.nickname)
client.message("Hello!")
})
}).catch(e => {
console.log("Catched an error!")
console.error(e)
})
Quickstart
Connecting to a TeamSpeak Server
There are 2 ways to connect with the TeamSpeak Server: using the wrapper TeamSpeak.connect() or by instanciating the TeamSpeak class by yourself
to connect with TeamSpeak.connect():
import { TeamSpeak } from "ts3-nodejs-library"
TeamSpeak.connect({
host: "127.0.0.1",
queryport: 10011
}).then(teamspeak => {
//you are now connected
}).catch(e => {
//an error occured during connecting
})
when instanciating it by yourself:
import { TeamSpeak } from "ts3-nodejs-library"
const teamspeak = new TeamSpeak({
host: "127.0.0.1",
queryport: 10011
})
teamspeak.on("ready", () => {
//teamspeak connected successfully
})
teamspeak.on("error", () => {
//teamspeak had an error
})
Configuration
| Parameter | Default | Description |
|---|---|---|
| host | “127.0.0.1” | hostname to connect to |
| queryport | 10011 | queryport to connect to |
| protocol | “raw” | either use telnet or ssh to connect to the server |
| serverport | empty |
the server port to select when connecting |
| username | empty |
the username to login with (required when using ssh) |
| password | empty |
the password to login with (required when using ssh) |
| nickname | empty |
the nickname to connect with when selecting a server |
| readyTimeout | 10000 | timeout in ms to wait for the connection to be built |
| keepAlive | true | wether a keepalive should be sent to the teamspeak server |
| localAddress | empty |
local address the socket should connect from |
Error handling when connecting
the method TeamSpeak.connect() will do error handling for you, when the query fails to connect or some of the commands which are being initially used to connect may encounter an error then the query will disconnect and reject the Promise with an error
However when instanciating TeamSpeak by yourself an error event might get thrown but the ready event will never fire
thats why TeamSpeak.connect() is more preferable to build a connection
Basic Usage
Sending a command
Sending a command is simple, every command will return a Promise. If you do not know what a Promise is then please read the documentation first: developer.mozilla.org > Promise
When using NodeJS then Promises are essential for further progress
Using Promises
teamspeak.whoami().then(whoami => {
console.log(whoami)
})
Using Async/Await
(async () => {
const whoami = await teamspeak.whoami()
console.log(whoami)
})()
whoami will then give you an object like
{
virtualserver_status: "online",
virtualserver_unique_identifier: "t1lytXTeyvmHXvNJ4ZcBBd15ugs=",
virtualserver_port: 9987,
virtualserver_id: 1,
client_id: 2,
client_channel_id: 1,
client_nickname: "serveradmin",
client_database_id: 0,
client_login_name: "serveradmin",
client_unique_identifier: "serveradmin",
client_origin_server_id: 0
}
Events
You can use teamspeak.on() to register to various events, you can find a list of all events here: https://multivit4min.github.io/TS3-NodeJS-Library/classes/teamspeak.html#on Note: with 3.x its not necessary to subscribe to events anymore, this will be done automatically now!
teamspeak.on("textmessage", ev => {
console.log(`${ev.invoker.nickname} just send the message "${ev.msg}"`)
})
Flood Protection
Flooding will be handled automatically.
When the Query gets accused of Flooding then it will return error with id 524 and an error message which states how much time needs to be waited.
This will be parsed automatically and the Query will wait for the given time (normally its 1 second) + 100 additional milliseconds (sometimes it happens the query gets banned when still sending too early)
Reconnecting
With version 2.3 this library is able to reconnect to its TeamSpeak Server when the connection has been lost. It restores its full context this includes:
- selecting the server
- logging in with the last credentials
- subscribing to all used events
- selecting the correct nickname
all commands which have been added in meantime while the teamspeak server was not connected will be still executed after and all pending commands will be sent AFTER connecting and restoring the context
an example on how this looks like:
import { TeamSpeak, QueryProtocol } from "./src/TeamSpeak"
TeamSpeak.connect({
host: "127.0.0.1",
queryport: 10011,
serverport: 9987,
protocol: QueryProtocol.RAW,
username: "serveradmin",
password: "xxx",
nickname: "test"
}).then(async teamspeak => {
teamspeak.on("close", async () => {
console.log("disconnected, trying to reconnect...")
await teamspeak.reconnect(-1, 1000)
console.log("reconnected!")
})
})
Update Notes from 1.x to 2.x
With version 2.x support for Client Events has been dropped instead use the events from the main class TeamSpeak. Additionally it comes with TeamSpeak.connect() in order to use a promise to connect to a teamspeak server. Multiple node methods have been replaced with a getter for ex: client.getDBID() -> client.databaseId, client.getUID() -> client.uniqueIdentifier, channel.getID() -> channel.cid The testing environment now runs via jest which makes mocking and testing easier. Since this project now is written in TypeScript vscode should now be completely capable to autocomplete, so there is no need to update docs on @types. Documentation software has been switched from documentation to typedoc
Update Notes to 2.3
The close event now only gets fired when a connection has been successfully established first! In order to get errors when connecting to a server use the error event instead. This was required in order to implement the reconnect logic.
Update Notes to 3.0
Some Parameters are now strings instead of numbers
With the free Beta TeamSpeak Servers for the TeamSpeak 5 Client there are IDs which use a 64 bit format. Since JavaScript starts to round at 53 bits those IDs will not be displayed correctly. In order to compensate this all IDs are now strings instead of numbers!
Function renames
Renamed some function in order to comply with JavaScript Standard
TeamSpeak#getClientByID -> TeamSpeak#getClientById
TeamSpeak#getClientByUID -> TeamSpeak#getClientByUid
TeamSpeak#getClientByDBID -> TeamSpeak#getClientByDbid
…
TeamSpeak#getChannelByID -> TeamSpeak#getChannelById\
Update to Permissions
Permissions will now be handled differently, if you for example want to add a Permission onto a ServerGroup then you can use
const group = await teamspeak.getServerGroupById(10)
if (!group) throw new Error("could not find group with id 10")
//old await teamspeak.serverGroupAddPerm(10, "i_channel_subscribe_power", 10, 0, 1)
await teamspeak.serverGroupAddPerm(group, {
permname: "i_channel_subscribe_power",
permvalue: 10,
skip: false,
negate: true
})
//or alternatively you can use it via the permission object
await teamspeak.serverGroupAddPerm(group)
.perm("i_channel_subscribe_power")
.value(10)
.skip(false)
.negate(true)
.update()
Permission List commands will now not give back a raw object but will give you an array of Permission class which you can dynamically update after, for example if you want to add all existing permissions the skip flag
const group = await teamspeak.getServerGroupById(10)
if (!group) throw new Error("could not find group with id 10")
const permlist = await group.permList()
await Promise.all(permlist.map(perm => perm.skip(true).update()))
or if you want to remove all permissions:
const group = await teamspeak.getServerGroupById(10)
if (!group) throw new Error("could not find group with id 10")
const permlist = await group.permList()
await Promise.all(permlist.map(perm => perm.remove()))
To retrieve the permission name or value you can use perm.getValue(), perm.getPerm(), perm.getSkip(), perm.getNegate()
Update to all parameters
all Parameters are now returned as camelcase and require camelcase characters in object properties
console.log(await teamspeak.whoami())
/**
* with < 3.0 it looked like:
* {
* virtualserver_status: "unknown",
* virtualserver_unique_identifier: undefined,
* virtualserver_port: 0,
* virtualserver_id: 0,
* client_id: 0,
* client_channel_id: 0,
* client_nickname: undefined,
* client_database_id: 0,
* client_login_name: undefined,
* client_unique_identifier: undefined,
* client_origin_server_id: 0
* }
* with converted to camelcase the response will look like:
* {
* virtualserverStatus: "unknown",
* virtualserverUniqueIdentifier: undefined,
* virtualserverPort: 0,
* virtualserverId: 0,
* clientId: 0,
* clientChannelId: 0,
* clientNickname: undefined,
* clientDatabaseId: 0,
* clientLoginName: undefined,
* clientUniqueIdentifier: undefined,
* clientOriginServerId: 0
* }
*/
the same is for parameters given to update certain things for example:
const channel = await teamspeak.getChannelById(10)
if (!channel) throw new Error("could not find channel with id 10")
//with version < 3.0
channel.edit({
channel_name: "foo",
channel_password: "bar",
channel_description: "lorem ipsum"
})
//with version >= 3.0
channel.edit({
channelName: "foo",
channelPassword: "bar",
channelDescription: "lorem ipsum"
})
In favor to have TeamSpeak#uploadIcon() a new dependency has been added buffer-crc32 Upload Icon takes as first argument the icon buffer and returns after a finnished upload the crc32 value of the buffer
Events
With 3.0 you do not need to subscribe to server events manually anymore! This will now done automatically when necessary! So you can remove those lines from your code:
Promise.all([
teamspeak.registerEvent("server"),
teamspeak.registerEvent("channel", 0),
teamspeak.registerEvent("textserver"),
teamspeak.registerEvent("textchannel"),
teamspeak.registerEvent("textprivate")
])
Authors
- David Kartnaller (Multivit4min) - Initial work
- Pascal Sthamer (P4sca1) and Mattis Krämer (Mattzi) - TypeScript typings
- Alexander (xIAlexanderIx)
See also the list of contributors who participated in this project.
Download Details:
Author: Multivit4min
Demo: https://multivit4min.github.io/TS3-NodeJS-Library/
Source Code: https://github.com/Multivit4min/TS3-NodeJS-Library
#nodejs #javascript #node
What is GEEK
Buddha Community
1653475560
A Pure PHP Implementation Of The MessagePack Serialization Format
msgpack.php
A pure PHP implementation of the MessagePack serialization format.
Features
- Fully compliant with the latest MessagePack specification
- Supports streaming unpacking
- Supports unsigned 64-bit integers handling
- Supports object serialization
- Fully tested
- Relatively fast
Installation
The recommended way to install the library is through Composer:
composer require rybakit/msgpack
Usage
Packing
To pack values you can either use an instance of a Packer:
$packer = new Packer();
$packed = $packer->pack($value);
or call a static method on the MessagePack class:
$packed = MessagePack::pack($value);
In the examples above, the method pack automatically packs a value depending on its type. However, not all PHP types can be uniquely translated to MessagePack types. For example, the MessagePack format defines map and array types, which are represented by a single array type in PHP. By default, the packer will pack a PHP array as a MessagePack array if it has sequential numeric keys, starting from 0 and as a MessagePack map otherwise:
$mpArr1 = $packer->pack([1, 2]); // MP array [1, 2]
$mpArr2 = $packer->pack([0 => 1, 1 => 2]); // MP array [1, 2]
$mpMap1 = $packer->pack([0 => 1, 2 => 3]); // MP map {0: 1, 2: 3}
$mpMap2 = $packer->pack([1 => 2, 2 => 3]); // MP map {1: 2, 2: 3}
$mpMap3 = $packer->pack(['a' => 1, 'b' => 2]); // MP map {a: 1, b: 2}
However, sometimes you need to pack a sequential array as a MessagePack map. To do this, use the packMap method:
$mpMap = $packer->packMap([1, 2]); // {0: 1, 1: 2}
Here is a list of type-specific packing methods:
$packer->packNil(); // MP nil
$packer->packBool(true); // MP bool
$packer->packInt(42); // MP int
$packer->packFloat(M_PI); // MP float (32 or 64)
$packer->packFloat32(M_PI); // MP float 32
$packer->packFloat64(M_PI); // MP float 64
$packer->packStr('foo'); // MP str
$packer->packBin("\x80"); // MP bin
$packer->packArray([1, 2]); // MP array
$packer->packMap(['a' => 1]); // MP map
$packer->packExt(1, "\xaa"); // MP ext
Check the "Custom types" section below on how to pack custom types.
Packing options
The Packer object supports a number of bitmask-based options for fine-tuning the packing process (defaults are in bold):
| Name | Description |
|---|---|
FORCE_STR | Forces PHP strings to be packed as MessagePack UTF-8 strings |
FORCE_BIN | Forces PHP strings to be packed as MessagePack binary data |
DETECT_STR_BIN | Detects MessagePack str/bin type automatically |
FORCE_ARR | Forces PHP arrays to be packed as MessagePack arrays |
FORCE_MAP | Forces PHP arrays to be packed as MessagePack maps |
DETECT_ARR_MAP | Detects MessagePack array/map type automatically |
FORCE_FLOAT32 | Forces PHP floats to be packed as 32-bits MessagePack floats |
FORCE_FLOAT64 | Forces PHP floats to be packed as 64-bits MessagePack floats |
The type detection mode (
DETECT_STR_BIN/DETECT_ARR_MAP) adds some overhead which can be noticed when you pack large (16- and 32-bit) arrays or strings. However, if you know the value type in advance (for example, you only work with UTF-8 strings or/and associative arrays), you can eliminate this overhead by forcing the packer to use the appropriate type, which will save it from running the auto-detection routine. Another option is to explicitly specify the value type. The library provides 2 auxiliary classes for this,MapandBin. Check the "Custom types" section below for details.
Examples:
// detect str/bin type and pack PHP 64-bit floats (doubles) to MP 32-bit floats
$packer = new Packer(PackOptions::DETECT_STR_BIN | PackOptions::FORCE_FLOAT32);
// these will throw MessagePack\Exception\InvalidOptionException
$packer = new Packer(PackOptions::FORCE_STR | PackOptions::FORCE_BIN);
$packer = new Packer(PackOptions::FORCE_FLOAT32 | PackOptions::FORCE_FLOAT64);
Unpacking
To unpack data you can either use an instance of a BufferUnpacker:
$unpacker = new BufferUnpacker();
$unpacker->reset($packed);
$value = $unpacker->unpack();
or call a static method on the MessagePack class:
$value = MessagePack::unpack($packed);
If the packed data is received in chunks (e.g. when reading from a stream), use the tryUnpack method, which attempts to unpack data and returns an array of unpacked messages (if any) instead of throwing an InsufficientDataException:
while ($chunk = ...) {
$unpacker->append($chunk);
if ($messages = $unpacker->tryUnpack()) {
return $messages;
}
}
If you want to unpack from a specific position in a buffer, use seek:
$unpacker->seek(42); // set position equal to 42 bytes
$unpacker->seek(-8); // set position to 8 bytes before the end of the buffer
To skip bytes from the current position, use skip:
$unpacker->skip(10); // set position to 10 bytes ahead of the current position
To get the number of remaining (unread) bytes in the buffer:
$unreadBytesCount = $unpacker->getRemainingCount();
To check whether the buffer has unread data:
$hasUnreadBytes = $unpacker->hasRemaining();
If needed, you can remove already read data from the buffer by calling:
$releasedBytesCount = $unpacker->release();
With the read method you can read raw (packed) data:
$packedData = $unpacker->read(2); // read 2 bytes
Besides the above methods BufferUnpacker provides type-specific unpacking methods, namely:
$unpacker->unpackNil(); // PHP null
$unpacker->unpackBool(); // PHP bool
$unpacker->unpackInt(); // PHP int
$unpacker->unpackFloat(); // PHP float
$unpacker->unpackStr(); // PHP UTF-8 string
$unpacker->unpackBin(); // PHP binary string
$unpacker->unpackArray(); // PHP sequential array
$unpacker->unpackMap(); // PHP associative array
$unpacker->unpackExt(); // PHP MessagePack\Type\Ext object
Unpacking options
The BufferUnpacker object supports a number of bitmask-based options for fine-tuning the unpacking process (defaults are in bold):
| Name | Description |
|---|---|
BIGINT_AS_STR | Converts overflowed integers to strings [1] |
BIGINT_AS_GMP | Converts overflowed integers to GMP objects [2] |
BIGINT_AS_DEC | Converts overflowed integers to Decimal\Decimal objects [3] |
1. The binary MessagePack format has unsigned 64-bit as its largest integer data type, but PHP does not support such integers, which means that an overflow can occur during unpacking.
2. Make sure the GMP extension is enabled.
3. Make sure the Decimal extension is enabled.
Examples:
$packedUint64 = "\xcf"."\xff\xff\xff\xff"."\xff\xff\xff\xff";
$unpacker = new BufferUnpacker($packedUint64);
var_dump($unpacker->unpack()); // string(20) "18446744073709551615"
$unpacker = new BufferUnpacker($packedUint64, UnpackOptions::BIGINT_AS_GMP);
var_dump($unpacker->unpack()); // object(GMP) {...}
$unpacker = new BufferUnpacker($packedUint64, UnpackOptions::BIGINT_AS_DEC);
var_dump($unpacker->unpack()); // object(Decimal\Decimal) {...}
Custom types
In addition to the basic types, the library provides functionality to serialize and deserialize arbitrary types. This can be done in several ways, depending on your use case. Let's take a look at them.
Type objects
If you need to serialize an instance of one of your classes into one of the basic MessagePack types, the best way to do this is to implement the CanBePacked interface in the class. A good example of such a class is the Map type class that comes with the library. This type is useful when you want to explicitly specify that a given PHP array should be packed as a MessagePack map without triggering an automatic type detection routine:
$packer = new Packer();
$packedMap = $packer->pack(new Map([1, 2, 3]));
$packedArray = $packer->pack([1, 2, 3]);
More type examples can be found in the src/Type directory.
Type transformers
As with type objects, type transformers are only responsible for serializing values. They should be used when you need to serialize a value that does not implement the CanBePacked interface. Examples of such values could be instances of built-in or third-party classes that you don't own, or non-objects such as resources.
A transformer class must implement the CanPack interface. To use a transformer, it must first be registered in the packer. Here is an example of how to serialize PHP streams into the MessagePack bin format type using one of the supplied transformers, StreamTransformer:
$packer = new Packer(null, [new StreamTransformer()]);
$packedBin = $packer->pack(fopen('/path/to/file', 'r+'));
More type transformer examples can be found in the src/TypeTransformer directory.
Extensions
In contrast to the cases described above, extensions are intended to handle extension types and are responsible for both serialization and deserialization of values (types).
An extension class must implement the Extension interface. To use an extension, it must first be registered in the packer and the unpacker.
The MessagePack specification divides extension types into two groups: predefined and application-specific. Currently, there is only one predefined type in the specification, Timestamp.
Timestamp
The Timestamp extension type is a predefined type. Support for this type in the library is done through the TimestampExtension class. This class is responsible for handling Timestamp objects, which represent the number of seconds and optional adjustment in nanoseconds:
$timestampExtension = new TimestampExtension();
$packer = new Packer();
$packer = $packer->extendWith($timestampExtension);
$unpacker = new BufferUnpacker();
$unpacker = $unpacker->extendWith($timestampExtension);
$packedTimestamp = $packer->pack(Timestamp::now());
$timestamp = $unpacker->reset($packedTimestamp)->unpack();
$seconds = $timestamp->getSeconds();
$nanoseconds = $timestamp->getNanoseconds();
When using the MessagePack class, the Timestamp extension is already registered:
$packedTimestamp = MessagePack::pack(Timestamp::now());
$timestamp = MessagePack::unpack($packedTimestamp);
Application-specific extensions
In addition, the format can be extended with your own types. For example, to make the built-in PHP DateTime objects first-class citizens in your code, you can create a corresponding extension, as shown in the example. Please note, that custom extensions have to be registered with a unique extension ID (an integer from 0 to 127).
More extension examples can be found in the examples/MessagePack directory.
To learn more about how extension types can be useful, check out this article.
Exceptions
If an error occurs during packing/unpacking, a PackingFailedException or an UnpackingFailedException will be thrown, respectively. In addition, an InsufficientDataException can be thrown during unpacking.
An InvalidOptionException will be thrown in case an invalid option (or a combination of mutually exclusive options) is used.
Tests
Run tests as follows:
vendor/bin/phpunit
Also, if you already have Docker installed, you can run the tests in a docker container. First, create a container:
./dockerfile.sh | docker build -t msgpack -
The command above will create a container named msgpack with PHP 8.1 runtime. You may change the default runtime by defining the PHP_IMAGE environment variable:
PHP_IMAGE='php:8.0-cli' ./dockerfile.sh | docker build -t msgpack -
See a list of various images here.
Then run the unit tests:
docker run --rm -v $PWD:/msgpack -w /msgpack msgpack
Fuzzing
To ensure that the unpacking works correctly with malformed/semi-malformed data, you can use a testing technique called Fuzzing. The library ships with a help file (target) for PHP-Fuzzer and can be used as follows:
php-fuzzer fuzz tests/fuzz_buffer_unpacker.php
Performance
To check performance, run:
php -n -dzend_extension=opcache.so \
-dpcre.jit=1 -dopcache.enable=1 -dopcache.enable_cli=1 \
tests/bench.php
Example output
Filter: MessagePack\Tests\Perf\Filter\ListFilter
Rounds: 3
Iterations: 100000
=============================================
Test/Target Packer BufferUnpacker
---------------------------------------------
nil .................. 0.0030 ........ 0.0139
false ................ 0.0037 ........ 0.0144
true ................. 0.0040 ........ 0.0137
7-bit uint #1 ........ 0.0052 ........ 0.0120
7-bit uint #2 ........ 0.0059 ........ 0.0114
7-bit uint #3 ........ 0.0061 ........ 0.0119
5-bit sint #1 ........ 0.0067 ........ 0.0126
5-bit sint #2 ........ 0.0064 ........ 0.0132
5-bit sint #3 ........ 0.0066 ........ 0.0135
8-bit uint #1 ........ 0.0078 ........ 0.0200
8-bit uint #2 ........ 0.0077 ........ 0.0212
8-bit uint #3 ........ 0.0086 ........ 0.0203
16-bit uint #1 ....... 0.0111 ........ 0.0271
16-bit uint #2 ....... 0.0115 ........ 0.0260
16-bit uint #3 ....... 0.0103 ........ 0.0273
32-bit uint #1 ....... 0.0116 ........ 0.0326
32-bit uint #2 ....... 0.0118 ........ 0.0332
32-bit uint #3 ....... 0.0127 ........ 0.0325
64-bit uint #1 ....... 0.0140 ........ 0.0277
64-bit uint #2 ....... 0.0134 ........ 0.0294
64-bit uint #3 ....... 0.0134 ........ 0.0281
8-bit int #1 ......... 0.0086 ........ 0.0241
8-bit int #2 ......... 0.0089 ........ 0.0225
8-bit int #3 ......... 0.0085 ........ 0.0229
16-bit int #1 ........ 0.0118 ........ 0.0280
16-bit int #2 ........ 0.0121 ........ 0.0270
16-bit int #3 ........ 0.0109 ........ 0.0274
32-bit int #1 ........ 0.0128 ........ 0.0346
32-bit int #2 ........ 0.0118 ........ 0.0339
32-bit int #3 ........ 0.0135 ........ 0.0368
64-bit int #1 ........ 0.0138 ........ 0.0276
64-bit int #2 ........ 0.0132 ........ 0.0286
64-bit int #3 ........ 0.0137 ........ 0.0274
64-bit int #4 ........ 0.0180 ........ 0.0285
64-bit float #1 ...... 0.0134 ........ 0.0284
64-bit float #2 ...... 0.0125 ........ 0.0275
64-bit float #3 ...... 0.0126 ........ 0.0283
fix string #1 ........ 0.0035 ........ 0.0133
fix string #2 ........ 0.0094 ........ 0.0216
fix string #3 ........ 0.0094 ........ 0.0222
fix string #4 ........ 0.0091 ........ 0.0241
8-bit string #1 ...... 0.0122 ........ 0.0301
8-bit string #2 ...... 0.0118 ........ 0.0304
8-bit string #3 ...... 0.0119 ........ 0.0315
16-bit string #1 ..... 0.0150 ........ 0.0388
16-bit string #2 ..... 0.1545 ........ 0.1665
32-bit string ........ 0.1570 ........ 0.1756
wide char string #1 .. 0.0091 ........ 0.0236
wide char string #2 .. 0.0122 ........ 0.0313
8-bit binary #1 ...... 0.0100 ........ 0.0302
8-bit binary #2 ...... 0.0123 ........ 0.0324
8-bit binary #3 ...... 0.0126 ........ 0.0327
16-bit binary ........ 0.0168 ........ 0.0372
32-bit binary ........ 0.1588 ........ 0.1754
fix array #1 ......... 0.0042 ........ 0.0131
fix array #2 ......... 0.0294 ........ 0.0367
fix array #3 ......... 0.0412 ........ 0.0472
16-bit array #1 ...... 0.1378 ........ 0.1596
16-bit array #2 ........... S ............. S
32-bit array .............. S ............. S
complex array ........ 0.1865 ........ 0.2283
fix map #1 ........... 0.0725 ........ 0.1048
fix map #2 ........... 0.0319 ........ 0.0405
fix map #3 ........... 0.0356 ........ 0.0665
fix map #4 ........... 0.0465 ........ 0.0497
16-bit map #1 ........ 0.2540 ........ 0.3028
16-bit map #2 ............. S ............. S
32-bit map ................ S ............. S
complex map .......... 0.2372 ........ 0.2710
fixext 1 ............. 0.0283 ........ 0.0358
fixext 2 ............. 0.0291 ........ 0.0371
fixext 4 ............. 0.0302 ........ 0.0355
fixext 8 ............. 0.0288 ........ 0.0384
fixext 16 ............ 0.0293 ........ 0.0359
8-bit ext ............ 0.0302 ........ 0.0439
16-bit ext ........... 0.0334 ........ 0.0499
32-bit ext ........... 0.1845 ........ 0.1888
32-bit timestamp #1 .. 0.0337 ........ 0.0547
32-bit timestamp #2 .. 0.0335 ........ 0.0560
64-bit timestamp #1 .. 0.0371 ........ 0.0575
64-bit timestamp #2 .. 0.0374 ........ 0.0542
64-bit timestamp #3 .. 0.0356 ........ 0.0533
96-bit timestamp #1 .. 0.0362 ........ 0.0699
96-bit timestamp #2 .. 0.0381 ........ 0.0701
96-bit timestamp #3 .. 0.0367 ........ 0.0687
=============================================
Total 2.7618 4.0820
Skipped 4 4
Failed 0 0
Ignored 0 0
With JIT:
php -n -dzend_extension=opcache.so \
-dpcre.jit=1 -dopcache.jit_buffer_size=64M -dopcache.jit=tracing -dopcache.enable=1 -dopcache.enable_cli=1 \
tests/bench.php
Example output
Filter: MessagePack\Tests\Perf\Filter\ListFilter
Rounds: 3
Iterations: 100000
=============================================
Test/Target Packer BufferUnpacker
---------------------------------------------
nil .................. 0.0005 ........ 0.0054
false ................ 0.0004 ........ 0.0059
true ................. 0.0004 ........ 0.0059
7-bit uint #1 ........ 0.0010 ........ 0.0047
7-bit uint #2 ........ 0.0010 ........ 0.0046
7-bit uint #3 ........ 0.0010 ........ 0.0046
5-bit sint #1 ........ 0.0025 ........ 0.0046
5-bit sint #2 ........ 0.0023 ........ 0.0046
5-bit sint #3 ........ 0.0024 ........ 0.0045
8-bit uint #1 ........ 0.0043 ........ 0.0081
8-bit uint #2 ........ 0.0043 ........ 0.0079
8-bit uint #3 ........ 0.0041 ........ 0.0080
16-bit uint #1 ....... 0.0064 ........ 0.0095
16-bit uint #2 ....... 0.0064 ........ 0.0091
16-bit uint #3 ....... 0.0064 ........ 0.0094
32-bit uint #1 ....... 0.0085 ........ 0.0114
32-bit uint #2 ....... 0.0077 ........ 0.0122
32-bit uint #3 ....... 0.0077 ........ 0.0120
64-bit uint #1 ....... 0.0085 ........ 0.0159
64-bit uint #2 ....... 0.0086 ........ 0.0157
64-bit uint #3 ....... 0.0086 ........ 0.0158
8-bit int #1 ......... 0.0042 ........ 0.0080
8-bit int #2 ......... 0.0042 ........ 0.0080
8-bit int #3 ......... 0.0042 ........ 0.0081
16-bit int #1 ........ 0.0065 ........ 0.0095
16-bit int #2 ........ 0.0065 ........ 0.0090
16-bit int #3 ........ 0.0056 ........ 0.0085
32-bit int #1 ........ 0.0067 ........ 0.0107
32-bit int #2 ........ 0.0066 ........ 0.0106
32-bit int #3 ........ 0.0063 ........ 0.0104
64-bit int #1 ........ 0.0072 ........ 0.0162
64-bit int #2 ........ 0.0073 ........ 0.0174
64-bit int #3 ........ 0.0072 ........ 0.0164
64-bit int #4 ........ 0.0077 ........ 0.0161
64-bit float #1 ...... 0.0053 ........ 0.0135
64-bit float #2 ...... 0.0053 ........ 0.0135
64-bit float #3 ...... 0.0052 ........ 0.0135
fix string #1 ....... -0.0002 ........ 0.0044
fix string #2 ........ 0.0035 ........ 0.0067
fix string #3 ........ 0.0035 ........ 0.0077
fix string #4 ........ 0.0033 ........ 0.0078
8-bit string #1 ...... 0.0059 ........ 0.0110
8-bit string #2 ...... 0.0063 ........ 0.0121
8-bit string #3 ...... 0.0064 ........ 0.0124
16-bit string #1 ..... 0.0099 ........ 0.0146
16-bit string #2 ..... 0.1522 ........ 0.1474
32-bit string ........ 0.1511 ........ 0.1483
wide char string #1 .. 0.0039 ........ 0.0084
wide char string #2 .. 0.0073 ........ 0.0123
8-bit binary #1 ...... 0.0040 ........ 0.0112
8-bit binary #2 ...... 0.0075 ........ 0.0123
8-bit binary #3 ...... 0.0077 ........ 0.0129
16-bit binary ........ 0.0096 ........ 0.0145
32-bit binary ........ 0.1535 ........ 0.1479
fix array #1 ......... 0.0008 ........ 0.0061
fix array #2 ......... 0.0121 ........ 0.0165
fix array #3 ......... 0.0193 ........ 0.0222
16-bit array #1 ...... 0.0607 ........ 0.0479
16-bit array #2 ........... S ............. S
32-bit array .............. S ............. S
complex array ........ 0.0749 ........ 0.0824
fix map #1 ........... 0.0329 ........ 0.0431
fix map #2 ........... 0.0161 ........ 0.0189
fix map #3 ........... 0.0205 ........ 0.0262
fix map #4 ........... 0.0252 ........ 0.0205
16-bit map #1 ........ 0.1016 ........ 0.0927
16-bit map #2 ............. S ............. S
32-bit map ................ S ............. S
complex map .......... 0.1096 ........ 0.1030
fixext 1 ............. 0.0157 ........ 0.0161
fixext 2 ............. 0.0175 ........ 0.0183
fixext 4 ............. 0.0156 ........ 0.0185
fixext 8 ............. 0.0163 ........ 0.0184
fixext 16 ............ 0.0164 ........ 0.0182
8-bit ext ............ 0.0158 ........ 0.0207
16-bit ext ........... 0.0203 ........ 0.0219
32-bit ext ........... 0.1614 ........ 0.1539
32-bit timestamp #1 .. 0.0195 ........ 0.0249
32-bit timestamp #2 .. 0.0188 ........ 0.0260
64-bit timestamp #1 .. 0.0207 ........ 0.0281
64-bit timestamp #2 .. 0.0212 ........ 0.0291
64-bit timestamp #3 .. 0.0207 ........ 0.0295
96-bit timestamp #1 .. 0.0222 ........ 0.0358
96-bit timestamp #2 .. 0.0228 ........ 0.0353
96-bit timestamp #3 .. 0.0210 ........ 0.0319
=============================================
Total 1.6432 1.9674
Skipped 4 4
Failed 0 0
Ignored 0 0
You may change default benchmark settings by defining the following environment variables:
| Name | Default |
|---|---|
| MP_BENCH_TARGETS | pure_p,pure_u, see a list of available targets |
| MP_BENCH_ITERATIONS | 100_000 |
| MP_BENCH_DURATION | not set |
| MP_BENCH_ROUNDS | 3 |
| MP_BENCH_TESTS | -@slow, see a list of available tests |
For example:
export MP_BENCH_TARGETS=pure_p
export MP_BENCH_ITERATIONS=1000000
export MP_BENCH_ROUNDS=5
# a comma separated list of test names
export MP_BENCH_TESTS='complex array, complex map'
# or a group name
# export MP_BENCH_TESTS='-@slow' // @pecl_comp
# or a regexp
# export MP_BENCH_TESTS='/complex (array|map)/'
Another example, benchmarking both the library and the PECL extension:
MP_BENCH_TARGETS=pure_p,pure_u,pecl_p,pecl_u \
php -n -dextension=msgpack.so -dzend_extension=opcache.so \
-dpcre.jit=1 -dopcache.enable=1 -dopcache.enable_cli=1 \
tests/bench.php
Example output
Filter: MessagePack\Tests\Perf\Filter\ListFilter
Rounds: 3
Iterations: 100000
===========================================================================
Test/Target Packer BufferUnpacker msgpack_pack msgpack_unpack
---------------------------------------------------------------------------
nil .................. 0.0031 ........ 0.0141 ...... 0.0055 ........ 0.0064
false ................ 0.0039 ........ 0.0154 ...... 0.0056 ........ 0.0053
true ................. 0.0038 ........ 0.0139 ...... 0.0056 ........ 0.0044
7-bit uint #1 ........ 0.0061 ........ 0.0110 ...... 0.0059 ........ 0.0046
7-bit uint #2 ........ 0.0065 ........ 0.0119 ...... 0.0042 ........ 0.0029
7-bit uint #3 ........ 0.0054 ........ 0.0117 ...... 0.0045 ........ 0.0025
5-bit sint #1 ........ 0.0047 ........ 0.0103 ...... 0.0038 ........ 0.0022
5-bit sint #2 ........ 0.0048 ........ 0.0117 ...... 0.0038 ........ 0.0022
5-bit sint #3 ........ 0.0046 ........ 0.0102 ...... 0.0038 ........ 0.0023
8-bit uint #1 ........ 0.0063 ........ 0.0174 ...... 0.0039 ........ 0.0031
8-bit uint #2 ........ 0.0063 ........ 0.0167 ...... 0.0040 ........ 0.0029
8-bit uint #3 ........ 0.0063 ........ 0.0168 ...... 0.0039 ........ 0.0030
16-bit uint #1 ....... 0.0092 ........ 0.0222 ...... 0.0049 ........ 0.0030
16-bit uint #2 ....... 0.0096 ........ 0.0227 ...... 0.0042 ........ 0.0046
16-bit uint #3 ....... 0.0123 ........ 0.0274 ...... 0.0059 ........ 0.0051
32-bit uint #1 ....... 0.0136 ........ 0.0331 ...... 0.0060 ........ 0.0048
32-bit uint #2 ....... 0.0130 ........ 0.0336 ...... 0.0070 ........ 0.0048
32-bit uint #3 ....... 0.0127 ........ 0.0329 ...... 0.0051 ........ 0.0048
64-bit uint #1 ....... 0.0126 ........ 0.0268 ...... 0.0055 ........ 0.0049
64-bit uint #2 ....... 0.0135 ........ 0.0281 ...... 0.0052 ........ 0.0046
64-bit uint #3 ....... 0.0131 ........ 0.0274 ...... 0.0069 ........ 0.0044
8-bit int #1 ......... 0.0077 ........ 0.0236 ...... 0.0058 ........ 0.0044
8-bit int #2 ......... 0.0087 ........ 0.0244 ...... 0.0058 ........ 0.0048
8-bit int #3 ......... 0.0084 ........ 0.0241 ...... 0.0055 ........ 0.0049
16-bit int #1 ........ 0.0112 ........ 0.0271 ...... 0.0048 ........ 0.0045
16-bit int #2 ........ 0.0124 ........ 0.0292 ...... 0.0057 ........ 0.0049
16-bit int #3 ........ 0.0118 ........ 0.0270 ...... 0.0058 ........ 0.0050
32-bit int #1 ........ 0.0137 ........ 0.0366 ...... 0.0058 ........ 0.0051
32-bit int #2 ........ 0.0133 ........ 0.0366 ...... 0.0056 ........ 0.0049
32-bit int #3 ........ 0.0129 ........ 0.0350 ...... 0.0052 ........ 0.0048
64-bit int #1 ........ 0.0145 ........ 0.0254 ...... 0.0034 ........ 0.0025
64-bit int #2 ........ 0.0097 ........ 0.0214 ...... 0.0034 ........ 0.0025
64-bit int #3 ........ 0.0096 ........ 0.0287 ...... 0.0059 ........ 0.0050
64-bit int #4 ........ 0.0143 ........ 0.0277 ...... 0.0059 ........ 0.0046
64-bit float #1 ...... 0.0134 ........ 0.0281 ...... 0.0057 ........ 0.0052
64-bit float #2 ...... 0.0141 ........ 0.0281 ...... 0.0057 ........ 0.0050
64-bit float #3 ...... 0.0144 ........ 0.0282 ...... 0.0057 ........ 0.0050
fix string #1 ........ 0.0036 ........ 0.0143 ...... 0.0066 ........ 0.0053
fix string #2 ........ 0.0107 ........ 0.0222 ...... 0.0065 ........ 0.0068
fix string #3 ........ 0.0116 ........ 0.0245 ...... 0.0063 ........ 0.0069
fix string #4 ........ 0.0105 ........ 0.0253 ...... 0.0083 ........ 0.0077
8-bit string #1 ...... 0.0126 ........ 0.0318 ...... 0.0075 ........ 0.0088
8-bit string #2 ...... 0.0121 ........ 0.0295 ...... 0.0076 ........ 0.0086
8-bit string #3 ...... 0.0125 ........ 0.0293 ...... 0.0130 ........ 0.0093
16-bit string #1 ..... 0.0159 ........ 0.0368 ...... 0.0117 ........ 0.0086
16-bit string #2 ..... 0.1547 ........ 0.1686 ...... 0.1516 ........ 0.1373
32-bit string ........ 0.1558 ........ 0.1729 ...... 0.1511 ........ 0.1396
wide char string #1 .. 0.0098 ........ 0.0237 ...... 0.0066 ........ 0.0065
wide char string #2 .. 0.0128 ........ 0.0291 ...... 0.0061 ........ 0.0082
8-bit binary #1 ........... I ............. I ........... F ............. I
8-bit binary #2 ........... I ............. I ........... F ............. I
8-bit binary #3 ........... I ............. I ........... F ............. I
16-bit binary ............. I ............. I ........... F ............. I
32-bit binary ............. I ............. I ........... F ............. I
fix array #1 ......... 0.0040 ........ 0.0129 ...... 0.0120 ........ 0.0058
fix array #2 ......... 0.0279 ........ 0.0390 ...... 0.0143 ........ 0.0165
fix array #3 ......... 0.0415 ........ 0.0463 ...... 0.0162 ........ 0.0187
16-bit array #1 ...... 0.1349 ........ 0.1628 ...... 0.0334 ........ 0.0341
16-bit array #2 ........... S ............. S ........... S ............. S
32-bit array .............. S ............. S ........... S ............. S
complex array ............. I ............. I ........... F ............. F
fix map #1 ................ I ............. I ........... F ............. I
fix map #2 ........... 0.0345 ........ 0.0391 ...... 0.0143 ........ 0.0168
fix map #3 ................ I ............. I ........... F ............. I
fix map #4 ........... 0.0459 ........ 0.0473 ...... 0.0151 ........ 0.0163
16-bit map #1 ........ 0.2518 ........ 0.2962 ...... 0.0400 ........ 0.0490
16-bit map #2 ............. S ............. S ........... S ............. S
32-bit map ................ S ............. S ........... S ............. S
complex map .......... 0.2380 ........ 0.2682 ...... 0.0545 ........ 0.0579
fixext 1 .................. I ............. I ........... F ............. F
fixext 2 .................. I ............. I ........... F ............. F
fixext 4 .................. I ............. I ........... F ............. F
fixext 8 .................. I ............. I ........... F ............. F
fixext 16 ................. I ............. I ........... F ............. F
8-bit ext ................. I ............. I ........... F ............. F
16-bit ext ................ I ............. I ........... F ............. F
32-bit ext ................ I ............. I ........... F ............. F
32-bit timestamp #1 ....... I ............. I ........... F ............. F
32-bit timestamp #2 ....... I ............. I ........... F ............. F
64-bit timestamp #1 ....... I ............. I ........... F ............. F
64-bit timestamp #2 ....... I ............. I ........... F ............. F
64-bit timestamp #3 ....... I ............. I ........... F ............. F
96-bit timestamp #1 ....... I ............. I ........... F ............. F
96-bit timestamp #2 ....... I ............. I ........... F ............. F
96-bit timestamp #3 ....... I ............. I ........... F ............. F
===========================================================================
Total 1.5625 2.3866 0.7735 0.7243
Skipped 4 4 4 4
Failed 0 0 24 17
Ignored 24 24 0 7
With JIT:
MP_BENCH_TARGETS=pure_p,pure_u,pecl_p,pecl_u \
php -n -dextension=msgpack.so -dzend_extension=opcache.so \
-dpcre.jit=1 -dopcache.jit_buffer_size=64M -dopcache.jit=tracing -dopcache.enable=1 -dopcache.enable_cli=1 \
tests/bench.php
Example output
Filter: MessagePack\Tests\Perf\Filter\ListFilter
Rounds: 3
Iterations: 100000
===========================================================================
Test/Target Packer BufferUnpacker msgpack_pack msgpack_unpack
---------------------------------------------------------------------------
nil .................. 0.0001 ........ 0.0052 ...... 0.0053 ........ 0.0042
false ................ 0.0007 ........ 0.0060 ...... 0.0057 ........ 0.0043
true ................. 0.0008 ........ 0.0060 ...... 0.0056 ........ 0.0041
7-bit uint #1 ........ 0.0031 ........ 0.0046 ...... 0.0062 ........ 0.0041
7-bit uint #2 ........ 0.0021 ........ 0.0043 ...... 0.0062 ........ 0.0041
7-bit uint #3 ........ 0.0022 ........ 0.0044 ...... 0.0061 ........ 0.0040
5-bit sint #1 ........ 0.0030 ........ 0.0048 ...... 0.0062 ........ 0.0040
5-bit sint #2 ........ 0.0032 ........ 0.0046 ...... 0.0062 ........ 0.0040
5-bit sint #3 ........ 0.0031 ........ 0.0046 ...... 0.0062 ........ 0.0040
8-bit uint #1 ........ 0.0054 ........ 0.0079 ...... 0.0062 ........ 0.0050
8-bit uint #2 ........ 0.0051 ........ 0.0079 ...... 0.0064 ........ 0.0044
8-bit uint #3 ........ 0.0051 ........ 0.0082 ...... 0.0062 ........ 0.0044
16-bit uint #1 ....... 0.0077 ........ 0.0094 ...... 0.0065 ........ 0.0045
16-bit uint #2 ....... 0.0077 ........ 0.0094 ...... 0.0063 ........ 0.0045
16-bit uint #3 ....... 0.0077 ........ 0.0095 ...... 0.0064 ........ 0.0047
32-bit uint #1 ....... 0.0088 ........ 0.0119 ...... 0.0063 ........ 0.0043
32-bit uint #2 ....... 0.0089 ........ 0.0117 ...... 0.0062 ........ 0.0039
32-bit uint #3 ....... 0.0089 ........ 0.0118 ...... 0.0063 ........ 0.0044
64-bit uint #1 ....... 0.0097 ........ 0.0155 ...... 0.0063 ........ 0.0045
64-bit uint #2 ....... 0.0095 ........ 0.0153 ...... 0.0061 ........ 0.0045
64-bit uint #3 ....... 0.0096 ........ 0.0156 ...... 0.0063 ........ 0.0047
8-bit int #1 ......... 0.0053 ........ 0.0083 ...... 0.0062 ........ 0.0044
8-bit int #2 ......... 0.0052 ........ 0.0080 ...... 0.0062 ........ 0.0044
8-bit int #3 ......... 0.0052 ........ 0.0080 ...... 0.0062 ........ 0.0043
16-bit int #1 ........ 0.0089 ........ 0.0097 ...... 0.0069 ........ 0.0046
16-bit int #2 ........ 0.0075 ........ 0.0093 ...... 0.0063 ........ 0.0043
16-bit int #3 ........ 0.0075 ........ 0.0094 ...... 0.0062 ........ 0.0046
32-bit int #1 ........ 0.0086 ........ 0.0122 ...... 0.0063 ........ 0.0044
32-bit int #2 ........ 0.0087 ........ 0.0120 ...... 0.0066 ........ 0.0046
32-bit int #3 ........ 0.0086 ........ 0.0121 ...... 0.0060 ........ 0.0044
64-bit int #1 ........ 0.0096 ........ 0.0149 ...... 0.0060 ........ 0.0045
64-bit int #2 ........ 0.0096 ........ 0.0157 ...... 0.0062 ........ 0.0044
64-bit int #3 ........ 0.0096 ........ 0.0160 ...... 0.0063 ........ 0.0046
64-bit int #4 ........ 0.0097 ........ 0.0157 ...... 0.0061 ........ 0.0044
64-bit float #1 ...... 0.0079 ........ 0.0153 ...... 0.0056 ........ 0.0044
64-bit float #2 ...... 0.0079 ........ 0.0152 ...... 0.0057 ........ 0.0045
64-bit float #3 ...... 0.0079 ........ 0.0155 ...... 0.0057 ........ 0.0044
fix string #1 ........ 0.0010 ........ 0.0045 ...... 0.0071 ........ 0.0044
fix string #2 ........ 0.0048 ........ 0.0075 ...... 0.0070 ........ 0.0060
fix string #3 ........ 0.0048 ........ 0.0086 ...... 0.0068 ........ 0.0060
fix string #4 ........ 0.0050 ........ 0.0088 ...... 0.0070 ........ 0.0059
8-bit string #1 ...... 0.0081 ........ 0.0129 ...... 0.0069 ........ 0.0062
8-bit string #2 ...... 0.0086 ........ 0.0128 ...... 0.0069 ........ 0.0065
8-bit string #3 ...... 0.0086 ........ 0.0126 ...... 0.0115 ........ 0.0065
16-bit string #1 ..... 0.0105 ........ 0.0137 ...... 0.0128 ........ 0.0068
16-bit string #2 ..... 0.1510 ........ 0.1486 ...... 0.1526 ........ 0.1391
32-bit string ........ 0.1517 ........ 0.1475 ...... 0.1504 ........ 0.1370
wide char string #1 .. 0.0044 ........ 0.0085 ...... 0.0067 ........ 0.0057
wide char string #2 .. 0.0081 ........ 0.0125 ...... 0.0069 ........ 0.0063
8-bit binary #1 ........... I ............. I ........... F ............. I
8-bit binary #2 ........... I ............. I ........... F ............. I
8-bit binary #3 ........... I ............. I ........... F ............. I
16-bit binary ............. I ............. I ........... F ............. I
32-bit binary ............. I ............. I ........... F ............. I
fix array #1 ......... 0.0014 ........ 0.0059 ...... 0.0132 ........ 0.0055
fix array #2 ......... 0.0146 ........ 0.0156 ...... 0.0155 ........ 0.0148
fix array #3 ......... 0.0211 ........ 0.0229 ...... 0.0179 ........ 0.0180
16-bit array #1 ...... 0.0673 ........ 0.0498 ...... 0.0343 ........ 0.0388
16-bit array #2 ........... S ............. S ........... S ............. S
32-bit array .............. S ............. S ........... S ............. S
complex array ............. I ............. I ........... F ............. F
fix map #1 ................ I ............. I ........... F ............. I
fix map #2 ........... 0.0148 ........ 0.0180 ...... 0.0156 ........ 0.0179
fix map #3 ................ I ............. I ........... F ............. I
fix map #4 ........... 0.0252 ........ 0.0201 ...... 0.0214 ........ 0.0167
16-bit map #1 ........ 0.1027 ........ 0.0836 ...... 0.0388 ........ 0.0510
16-bit map #2 ............. S ............. S ........... S ............. S
32-bit map ................ S ............. S ........... S ............. S
complex map .......... 0.1104 ........ 0.1010 ...... 0.0556 ........ 0.0602
fixext 1 .................. I ............. I ........... F ............. F
fixext 2 .................. I ............. I ........... F ............. F
fixext 4 .................. I ............. I ........... F ............. F
fixext 8 .................. I ............. I ........... F ............. F
fixext 16 ................. I ............. I ........... F ............. F
8-bit ext ................. I ............. I ........... F ............. F
16-bit ext ................ I ............. I ........... F ............. F
32-bit ext ................ I ............. I ........... F ............. F
32-bit timestamp #1 ....... I ............. I ........... F ............. F
32-bit timestamp #2 ....... I ............. I ........... F ............. F
64-bit timestamp #1 ....... I ............. I ........... F ............. F
64-bit timestamp #2 ....... I ............. I ........... F ............. F
64-bit timestamp #3 ....... I ............. I ........... F ............. F
96-bit timestamp #1 ....... I ............. I ........... F ............. F
96-bit timestamp #2 ....... I ............. I ........... F ............. F
96-bit timestamp #3 ....... I ............. I ........... F ............. F
===========================================================================
Total 0.9642 1.0909 0.8224 0.7213
Skipped 4 4 4 4
Failed 0 0 24 17
Ignored 24 24 0 7
Note that the msgpack extension (v2.1.2) doesn't support ext, bin and UTF-8 str types.
License
The library is released under the MIT License. See the bundled LICENSE file for details.
Author: rybakit
Source Code: https://github.com/rybakit/msgpack.php
License: MIT License
1669952228
Dijkstra's Algorithm Explained with Examples
In this tutorial, you'll learn: What is Dijkstra's Algorithm and how Dijkstra's algorithm works with the help of visual guides.
You can use algorithms in programming to solve specific problems through a set of precise instructions or procedures.
Dijkstra's algorithm is one of many graph algorithms you'll come across. It is used to find the shortest path from a fixed node to all other nodes in a graph.
There are different representations of Dijkstra's algorithm. You can either find the shortest path between two nodes, or the shortest path from a fixed node to the rest of the nodes in a graph.
In this article, you'll learn how Dijkstra's algorithm works with the help of visual guides.
How Does Dijkstra’s Algorithm Work?
Before we dive into more detailed visual examples, you need to understand how Dijkstra's algorithm works.
Although the theoretical explanation may seem a bit abstract, it'll help you understand the practical aspect better.
In a given graph containing different nodes, we are required to get the shortest path from a given node to the rest of the nodes.
These nodes can represent any object like the names of cities, letters, and so on.
Between each node is a number denoting the distance between two nodes, as you can see in the image below:
We usually work with two arrays – one for visited nodes, and another for unvisited nodes. You'll learn more about the arrays in the next section.
When a node is visited, the algorithm calculates how long it took to get to the node and stores the distance. If a shorter path to a node is found, the initial value assigned for the distance is updated.
Note that a node cannot be visited twice.
The algorithm runs recursively until all the nodes have been visited.
Dijkstra's Algorithm Example
In this section, we'll take a look at a practical example that shows how Dijkstra's algorithm works.
Here's the graph we'll be working with:
We'll use the table below to put down the visited nodes and their distance from the fixed node:
| NODE | SHORTEST DISTANCE FROM FIXED NODE |
|---|---|
| A | ∞ |
| B | ∞ |
| C | ∞ |
| D | ∞ |
| E | ∞ |
Visited nodes = []
Unvisited nodes = [A,B,C,D,E]
Above, we have a table showing each node and the shortest distance from the that node to the fixed node. We are yet to choose the fixed node.
Note that the distance for each node in the table is currently denoted as infinity (∞). This is because we don't know the shortest distance yet.
We also have two arrays – visited and unvisited. Whenever a node is visited, it is added to the visited nodes array.
Let's get started!
To simplify things, I'll break the process down into iterations. You'll see what happens in each step with the aid of diagrams.
Iteration #1
The first iteration might seem confusing, but that's totally fine. Once we start repeating the process in each iteration, you'll have a clearer picture of how the algorithm works.
Step #1 - Pick an unvisited node
We'll choose A as the fixed node. So we'll find the shortest distance from A to every other node in the graph.
We're going to give A a distance of 0 because it is the initial node. So the table would look like this:
| NODE | SHORTEST DISTANCE FROM FIXED NODE |
|---|---|
| A | 0 |
| B | ∞ |
| C | ∞ |
| D | ∞ |
| E | ∞ |
Step #2 - Find the distance from current node
The next thing to do after choosing a node is to find the distance from it to the unvisited nodes around it.
The two unvisited nodes directly linked to A are B and C.
To get the distance from A to B:
0 + 4 = 4
0 being the value of the current node (A), and 4 being the distance between A and B in the graph.
To get the distance from A to C:
0 + 2 = 2
Step #3 - Update table with known distances
In the last step, we got 4 and 2 as the values of B and C respectively. So we'll update the table with those values:
| NODE | SHORTEST DISTANCE FROM FIXED NODE |
|---|---|
| A | 0 |
| B | 4 |
| C | 2 |
| D | ∞ |
| E | ∞ |
Step #4 - Update arrays
At this point, the first iteration is complete. We'll move node A to the visited nodes array:
Visited nodes = [A]
Unvisited nodes = [B,C,D,E]
Before we proceed to the next iteration, you should know the following:
- Once a node has been visited, it cannot be linked to the current node. Refer to step #2 in the iteration above and step #2 in the next iteration.
- A node cannot be visited twice.
- You can only update the shortest known distance if you get a value smaller than the recorded distance.
Iteration #2
Step #1 - Pick an unvisited node
We have four unvisited nodes — [B,C,D,E]. So how do you know which node to pick for the next iteration?
Well, we pick the node with the smallest known distance recorded in the table. Here's the table:
| NODE | SHORTEST DISTANCE FROM FIXED NODE |
|---|---|
| A | 0 |
| B | 4 |
| C | 2 |
| D | ∞ |
| E | ∞ |
So we're going with node C.
Step #2 - Find the distance from current node
To find the distance from the current node to the fixed node, we have to consider the nodes linked to the current node.
The nodes linked to the current node are A and B.
But A has been visited in the previous iteration so it will not be linked to the current node. That is:
From the diagram above,
- The green color denotes the current node.
- The blue color denotes the visited nodes. We cannot link to them or visit them again.
- The red color shows the link from the unvisited nodes to the current node.
To find the distance from C to B:
2 + 1 = 3
2 above is recorded distance for node C while 1 is the distance between C and B in the graph.
Step #3 - Update table with known distances
In the last step, we got the value of B to be 3. In the first iteration, it was 4.
We're going to update the distance in the table to 3.
| NODE | SHORTEST DISTANCE FROM FIXED NODE |
|---|---|
| A | 0 |
| B | 3 |
| C | 2 |
| D | ∞ |
| E | ∞ |
So, A --> B = 4 (First iteration).
A --> C --> B = 3 (Second iteration).
The algorithm has helped us find the shortest path to B from A.
Step #4 - Update arrays
We're done with the last visited node. Let's add it to the visited nodes array:
Visited nodes = [A,C]
Unvisited nodes = [B,D,E]
Iteration #3
Step #1 - Pick an unvisited node
We're down to three unvisited nodes — [B,D,E]. From the array, B has the shortest known distance.
To restate what is going on in the diagram above:
- The green color denotes the current node.
- The blue color denotes the visited nodes. We cannot link to them or visit them again.
- The red color shows the link from the unvisited nodes to the current node.
Step #2 - Find the distance from current node
The nodes linked to the current node are D and E.
B (the current node) has a value of 3. Therefore,
For node D, 3 + 3 = 6.
For node E, 3 + 2 = 5.
Step #3 - Update table with known distances
| NODE | SHORTEST DISTANCE FROM FIXED NODE |
|---|---|
| A | 0 |
| B | 3 |
| C | 2 |
| D | 6 |
| E | 5 |
Step #4 - Update arrays
Visited nodes = [A,C,B]
Unvisited nodes = [D,E]
Iteration #4
Step #1 - Pick an unvisited node
Like other iterations, we'll go with the unvisited node with the shortest known distance. That is E.
Step #2 - Find the distance from current node
According to our table, E has a value of 5.
For D in the current iteration,
5 + 5 = 10.
The value gotten for D here is 10, which is greater than the recorded value of 6 in the previous iteration. For this reason, we'll not update the table.
Step #3 - Update table with known distances
Our table remains the same:
| NODE | SHORTEST DISTANCE FROM FIXED NODE |
|---|---|
| A | 0 |
| B | 3 |
| C | 2 |
| D | 6 |
| E | 5 |
Step #4 - Update arrays
Visited nodes = [A,C,B,E]
Unvisited nodes = [D]
Iteration #5
Step #1 - Pick an unvisited node
We're currently left with one node in the unvisited array — D.
Step #2 - Find the distance from current node
The algorithm has gotten to the last iteration. This is because all nodes linked to the current node have been visited already so we can't link to them.
Step #3 - Update table with known distances
Our table remains the same:
| NODE | SHORTEST DISTANCE FROM FIXED NODE |
|---|---|
| A | 0 |
| B | 3 |
| C | 2 |
| D | 6 |
| E | 5 |
At this point, we have updated the table with the shortest distance from the fixed node to every other node in the graph.
Step #4 - Update arrays
Visited nodes = [A,C,B,E,D]
Unvisited nodes = []
As can be seen above, we have no nodes left to visit. Using Dijkstra's algorithm, we've found the shortest distance from the fixed node to others nodes in the graph.
Dijkstra's Algorithm Pseudocode Example
The pseudocode example in this section was gotten from Wikipedia. Here it is:
1 function Dijkstra(Graph, source):
2
3 for each vertex v in Graph.Vertices:
4 dist[v] ← INFINITY
5 prev[v] ← UNDEFINED
6 add v to Q
7 dist[source] ← 0
8
9 while Q is not empty:
10 u ← vertex in Q with min dist[u]
11 remove u from Q
12
13 for each neighbor v of u still in Q:
14 alt ← dist[u] + Graph.Edges(u, v)
15 if alt < dist[v]:
16 dist[v] ← alt
17 prev[v] ← u
18
19 return dist[], prev[]Applications of Dijkstra's Algorithm
Here are some of the common applications of Dijkstra's algorithm:
- In maps to get the shortest distance between locations. An example is Google Maps.
- In telecommunications to determine transmission rate.
- In robotic design to determine shortest path for automated robots.
Summary
In this article, we talked about Dijkstra's algorithm. It is used to find the shortest distance from a fixed node to all other nodes in a graph.
We started by giving a brief summary of how the algorithm works.
We then had a look at an example that further explained Dijkstra's algorithm in steps using visual guides.
We concluded with a pseudocode example and some of the applications of Dijkstra's algorithm.
Happy coding!
Original article source at https://www.freecodecamp.org
#algorithm #datastructures
1630743562
A Wrapper for Sembast and SQFlite to Enable Easy
FHIR_DB
This is really just a wrapper around Sembast_SQFLite - so all of the heavy lifting was done by Alex Tekartik. I highly recommend that if you have any questions about working with this package that you take a look at Sembast. He's also just a super nice guy, and even answered a question for me when I was deciding which sembast version to use. As usual, ResoCoder also has a good tutorial.
I have an interest in low-resource settings and thus a specific reason to be able to store data offline. To encourage this use, there are a number of other packages I have created based around the data format FHIR. FHIR® is the registered trademark of HL7 and is used with the permission of HL7. Use of the FHIR trademark does not constitute endorsement of this product by HL7.
Using the Db
So, while not absolutely necessary, I highly recommend that you use some sort of interface class. This adds the benefit of more easily handling errors, plus if you change to a different database in the future, you don't have to change the rest of your app, just the interface.
I've used something like this in my projects:
class IFhirDb {
IFhirDb();
final ResourceDao resourceDao = ResourceDao();
Future<Either<DbFailure, Resource>> save(Resource resource) async {
Resource resultResource;
try {
resultResource = await resourceDao.save(resource);
} catch (error) {
return left(DbFailure.unableToSave(error: error.toString()));
}
return right(resultResource);
}
Future<Either<DbFailure, List<Resource>>> returnListOfSingleResourceType(
String resourceType) async {
List<Resource> resultList;
try {
resultList =
await resourceDao.getAllSortedById(resourceType: resourceType);
} catch (error) {
return left(DbFailure.unableToObtainList(error: error.toString()));
}
return right(resultList);
}
Future<Either<DbFailure, List<Resource>>> searchFunction(
String resourceType, String searchString, String reference) async {
List<Resource> resultList;
try {
resultList =
await resourceDao.searchFor(resourceType, searchString, reference);
} catch (error) {
return left(DbFailure.unableToObtainList(error: error.toString()));
}
return right(resultList);
}
}
I like this because in case there's an i/o error or something, it won't crash your app. Then, you can call this interface in your app like the following:
final patient = Patient(
resourceType: 'Patient',
name: [HumanName(text: 'New Patient Name')],
birthDate: Date(DateTime.now()),
);
final saveResult = await IFhirDb().save(patient);
This will save your newly created patient to the locally embedded database.
IMPORTANT: this database will expect that all previously created resources have an id. When you save a resource, it will check to see if that resource type has already been stored. (Each resource type is saved in it's own store in the database). It will then check if there is an ID. If there's no ID, it will create a new one for that resource (along with metadata on version number and creation time). It will save it, and return the resource. If it already has an ID, it will copy the the old version of the resource into a _history store. It will then update the metadata of the new resource and save that version into the appropriate store for that resource. If, for instance, we have a previously created patient:
{
"resourceType": "Patient",
"id": "fhirfli-294057507-6811107",
"meta": {
"versionId": "1",
"lastUpdated": "2020-10-16T19:41:28.054369Z"
},
"name": [
{
"given": ["New"],
"family": "Patient"
}
],
"birthDate": "2020-10-16"
}
And we update the last name to 'Provider'. The above version of the patient will be kept in _history, while in the 'Patient' store in the db, we will have the updated version:
{
"resourceType": "Patient",
"id": "fhirfli-294057507-6811107",
"meta": {
"versionId": "2",
"lastUpdated": "2020-10-16T19:45:07.316698Z"
},
"name": [
{
"given": ["New"],
"family": "Provider"
}
],
"birthDate": "2020-10-16"
}
This way we can keep track of all previous version of all resources (which is obviously important in medicine).
For most of the interactions (saving, deleting, etc), they work the way you'd expect. The only difference is search. Because Sembast is NoSQL, we can search on any of the fields in a resource. If in our interface class, we have the following function:
Future<Either<DbFailure, List<Resource>>> searchFunction(
String resourceType, String searchString, String reference) async {
List<Resource> resultList;
try {
resultList =
await resourceDao.searchFor(resourceType, searchString, reference);
} catch (error) {
return left(DbFailure.unableToObtainList(error: error.toString()));
}
return right(resultList);
}
You can search for all immunizations of a certain patient:
searchFunction(
'Immunization', 'patient.reference', 'Patient/$patientId');
This function will search through all entries in the 'Immunization' store. It will look at all 'patient.reference' fields, and return any that match 'Patient/$patientId'.
The last thing I'll mention is that this is a password protected db, using AES-256 encryption (although it can also use Salsa20). Anytime you use the db, you have the option of using a password for encryption/decryption. Remember, if you setup the database using encryption, you will only be able to access it using that same password. When you're ready to change the password, you will need to call the update password function. If we again assume we created a change password method in our interface, it might look something like this:
class IFhirDb {
IFhirDb();
final ResourceDao resourceDao = ResourceDao();
...
Future<Either<DbFailure, Unit>> updatePassword(String oldPassword, String newPassword) async {
try {
await resourceDao.updatePw(oldPassword, newPassword);
} catch (error) {
return left(DbFailure.unableToUpdatePassword(error: error.toString()));
}
return right(Unit);
}
You don't have to use a password, and in that case, it will save the db file as plain text. If you want to add a password later, it will encrypt it at that time.
General Store
After using this for a while in an app, I've realized that it needs to be able to store data apart from just FHIR resources, at least on occasion. For this, I've added a second class for all versions of the database called GeneralDao. This is similar to the ResourceDao, but fewer options. So, in order to save something, it would look like this:
await GeneralDao().save('password', {'new':'map'});
await GeneralDao().save('password', {'new':'map'}, 'key');
The difference between these two options is that the first one will generate a key for the map being stored, while the second will store the map using the key provided. Both will return the key after successfully storing the map.
Other functions available include:
// deletes everything in the general store
await GeneralDao().deleteAllGeneral('password');
// delete specific entry
await GeneralDao().delete('password','key');
// returns map with that key
await GeneralDao().find('password', 'key');
FHIR® is a registered trademark of Health Level Seven International (HL7) and its use does not constitute an endorsement of products by HL7®
Use this package as a library
Depend on it
Run this command:
With Flutter:
$ flutter pub add fhir_dbThis will add a line like this to your package's pubspec.yaml (and run an implicit flutter pub get):
dependencies:
fhir_db: ^0.4.3Alternatively, your editor might support or flutter pub get. Check the docs for your editor to learn more.
Import it
Now in your Dart code, you can use:
import 'package:fhir_db/dstu2.dart';
import 'package:fhir_db/dstu2/fhir_db.dart';
import 'package:fhir_db/dstu2/general_dao.dart';
import 'package:fhir_db/dstu2/resource_dao.dart';
import 'package:fhir_db/encrypt/aes.dart';
import 'package:fhir_db/encrypt/salsa.dart';
import 'package:fhir_db/r4.dart';
import 'package:fhir_db/r4/fhir_db.dart';
import 'package:fhir_db/r4/general_dao.dart';
import 'package:fhir_db/r4/resource_dao.dart';
import 'package:fhir_db/r5.dart';
import 'package:fhir_db/r5/fhir_db.dart';
import 'package:fhir_db/r5/general_dao.dart';
import 'package:fhir_db/r5/resource_dao.dart';
import 'package:fhir_db/stu3.dart';
import 'package:fhir_db/stu3/fhir_db.dart';
import 'package:fhir_db/stu3/general_dao.dart';
import 'package:fhir_db/stu3/resource_dao.dart'; import 'package:fhir/r4.dart';
import 'package:fhir_db/r4.dart';
import 'package:flutter/material.dart';
import 'package:test/test.dart';
Future<void> main() async {
WidgetsFlutterBinding.ensureInitialized();
final resourceDao = ResourceDao();
// await resourceDao.updatePw('newPw', null);
await resourceDao.deleteAllResources(null);
group('Playing with passwords', () {
test('Playing with Passwords', () async {
final patient = Patient(id: Id('1'));
final saved = await resourceDao.save(null, patient);
await resourceDao.updatePw(null, 'newPw');
final search1 = await resourceDao.find('newPw',
resourceType: R4ResourceType.Patient, id: Id('1'));
expect(saved, search1[0]);
await resourceDao.updatePw('newPw', 'newerPw');
final search2 = await resourceDao.find('newerPw',
resourceType: R4ResourceType.Patient, id: Id('1'));
expect(saved, search2[0]);
await resourceDao.updatePw('newerPw', null);
final search3 = await resourceDao.find(null,
resourceType: R4ResourceType.Patient, id: Id('1'));
expect(saved, search3[0]);
await resourceDao.deleteAllResources(null);
});
});
final id = Id('12345');
group('Saving Things:', () {
test('Save Patient', () async {
final humanName = HumanName(family: 'Atreides', given: ['Duke']);
final patient = Patient(id: id, name: [humanName]);
final saved = await resourceDao.save(null, patient);
expect(saved.id, id);
expect((saved as Patient).name?[0], humanName);
});
test('Save Organization', () async {
final organization = Organization(id: id, name: 'FhirFli');
final saved = await resourceDao.save(null, organization);
expect(saved.id, id);
expect((saved as Organization).name, 'FhirFli');
});
test('Save Observation1', () async {
final observation1 = Observation(
id: Id('obs1'),
code: CodeableConcept(text: 'Observation #1'),
effectiveDateTime: FhirDateTime(DateTime(1981, 09, 18)),
);
final saved = await resourceDao.save(null, observation1);
expect(saved.id, Id('obs1'));
expect((saved as Observation).code.text, 'Observation #1');
});
test('Save Observation1 Again', () async {
final observation1 = Observation(
id: Id('obs1'),
code: CodeableConcept(text: 'Observation #1 - Updated'));
final saved = await resourceDao.save(null, observation1);
expect(saved.id, Id('obs1'));
expect((saved as Observation).code.text, 'Observation #1 - Updated');
expect(saved.meta?.versionId, Id('2'));
});
test('Save Observation2', () async {
final observation2 = Observation(
id: Id('obs2'),
code: CodeableConcept(text: 'Observation #2'),
effectiveDateTime: FhirDateTime(DateTime(1981, 09, 18)),
);
final saved = await resourceDao.save(null, observation2);
expect(saved.id, Id('obs2'));
expect((saved as Observation).code.text, 'Observation #2');
});
test('Save Observation3', () async {
final observation3 = Observation(
id: Id('obs3'),
code: CodeableConcept(text: 'Observation #3'),
effectiveDateTime: FhirDateTime(DateTime(1981, 09, 18)),
);
final saved = await resourceDao.save(null, observation3);
expect(saved.id, Id('obs3'));
expect((saved as Observation).code.text, 'Observation #3');
});
});
group('Finding Things:', () {
test('Find 1st Patient', () async {
final search = await resourceDao.find(null,
resourceType: R4ResourceType.Patient, id: id);
final humanName = HumanName(family: 'Atreides', given: ['Duke']);
expect(search.length, 1);
expect((search[0] as Patient).name?[0], humanName);
});
test('Find 3rd Observation', () async {
final search = await resourceDao.find(null,
resourceType: R4ResourceType.Observation, id: Id('obs3'));
expect(search.length, 1);
expect(search[0].id, Id('obs3'));
expect((search[0] as Observation).code.text, 'Observation #3');
});
test('Find All Observations', () async {
final search = await resourceDao.getResourceType(
null,
resourceTypes: [R4ResourceType.Observation],
);
expect(search.length, 3);
final idList = [];
for (final obs in search) {
idList.add(obs.id.toString());
}
expect(idList.contains('obs1'), true);
expect(idList.contains('obs2'), true);
expect(idList.contains('obs3'), true);
});
test('Find All (non-historical) Resources', () async {
final search = await resourceDao.getAll(null);
expect(search.length, 5);
final patList = search.toList();
final orgList = search.toList();
final obsList = search.toList();
patList.retainWhere(
(resource) => resource.resourceType == R4ResourceType.Patient);
orgList.retainWhere(
(resource) => resource.resourceType == R4ResourceType.Organization);
obsList.retainWhere(
(resource) => resource.resourceType == R4ResourceType.Observation);
expect(patList.length, 1);
expect(orgList.length, 1);
expect(obsList.length, 3);
});
});
group('Deleting Things:', () {
test('Delete 2nd Observation', () async {
await resourceDao.delete(
null, null, R4ResourceType.Observation, Id('obs2'), null, null);
final search = await resourceDao.getResourceType(
null,
resourceTypes: [R4ResourceType.Observation],
);
expect(search.length, 2);
final idList = [];
for (final obs in search) {
idList.add(obs.id.toString());
}
expect(idList.contains('obs1'), true);
expect(idList.contains('obs2'), false);
expect(idList.contains('obs3'), true);
});
test('Delete All Observations', () async {
await resourceDao.deleteSingleType(null,
resourceType: R4ResourceType.Observation);
final search = await resourceDao.getAll(null);
expect(search.length, 2);
final patList = search.toList();
final orgList = search.toList();
patList.retainWhere(
(resource) => resource.resourceType == R4ResourceType.Patient);
orgList.retainWhere(
(resource) => resource.resourceType == R4ResourceType.Organization);
expect(patList.length, 1);
expect(patList.length, 1);
});
test('Delete All Resources', () async {
await resourceDao.deleteAllResources(null);
final search = await resourceDao.getAll(null);
expect(search.length, 0);
});
});
group('Password - Saving Things:', () {
test('Save Patient', () async {
await resourceDao.updatePw(null, 'newPw');
final humanName = HumanName(family: 'Atreides', given: ['Duke']);
final patient = Patient(id: id, name: [humanName]);
final saved = await resourceDao.save('newPw', patient);
expect(saved.id, id);
expect((saved as Patient).name?[0], humanName);
});
test('Save Organization', () async {
final organization = Organization(id: id, name: 'FhirFli');
final saved = await resourceDao.save('newPw', organization);
expect(saved.id, id);
expect((saved as Organization).name, 'FhirFli');
});
test('Save Observation1', () async {
final observation1 = Observation(
id: Id('obs1'),
code: CodeableConcept(text: 'Observation #1'),
effectiveDateTime: FhirDateTime(DateTime(1981, 09, 18)),
);
final saved = await resourceDao.save('newPw', observation1);
expect(saved.id, Id('obs1'));
expect((saved as Observation).code.text, 'Observation #1');
});
test('Save Observation1 Again', () async {
final observation1 = Observation(
id: Id('obs1'),
code: CodeableConcept(text: 'Observation #1 - Updated'));
final saved = await resourceDao.save('newPw', observation1);
expect(saved.id, Id('obs1'));
expect((saved as Observation).code.text, 'Observation #1 - Updated');
expect(saved.meta?.versionId, Id('2'));
});
test('Save Observation2', () async {
final observation2 = Observation(
id: Id('obs2'),
code: CodeableConcept(text: 'Observation #2'),
effectiveDateTime: FhirDateTime(DateTime(1981, 09, 18)),
);
final saved = await resourceDao.save('newPw', observation2);
expect(saved.id, Id('obs2'));
expect((saved as Observation).code.text, 'Observation #2');
});
test('Save Observation3', () async {
final observation3 = Observation(
id: Id('obs3'),
code: CodeableConcept(text: 'Observation #3'),
effectiveDateTime: FhirDateTime(DateTime(1981, 09, 18)),
);
final saved = await resourceDao.save('newPw', observation3);
expect(saved.id, Id('obs3'));
expect((saved as Observation).code.text, 'Observation #3');
});
});
group('Password - Finding Things:', () {
test('Find 1st Patient', () async {
final search = await resourceDao.find('newPw',
resourceType: R4ResourceType.Patient, id: id);
final humanName = HumanName(family: 'Atreides', given: ['Duke']);
expect(search.length, 1);
expect((search[0] as Patient).name?[0], humanName);
});
test('Find 3rd Observation', () async {
final search = await resourceDao.find('newPw',
resourceType: R4ResourceType.Observation, id: Id('obs3'));
expect(search.length, 1);
expect(search[0].id, Id('obs3'));
expect((search[0] as Observation).code.text, 'Observation #3');
});
test('Find All Observations', () async {
final search = await resourceDao.getResourceType(
'newPw',
resourceTypes: [R4ResourceType.Observation],
);
expect(search.length, 3);
final idList = [];
for (final obs in search) {
idList.add(obs.id.toString());
}
expect(idList.contains('obs1'), true);
expect(idList.contains('obs2'), true);
expect(idList.contains('obs3'), true);
});
test('Find All (non-historical) Resources', () async {
final search = await resourceDao.getAll('newPw');
expect(search.length, 5);
final patList = search.toList();
final orgList = search.toList();
final obsList = search.toList();
patList.retainWhere(
(resource) => resource.resourceType == R4ResourceType.Patient);
orgList.retainWhere(
(resource) => resource.resourceType == R4ResourceType.Organization);
obsList.retainWhere(
(resource) => resource.resourceType == R4ResourceType.Observation);
expect(patList.length, 1);
expect(orgList.length, 1);
expect(obsList.length, 3);
});
});
group('Password - Deleting Things:', () {
test('Delete 2nd Observation', () async {
await resourceDao.delete(
'newPw', null, R4ResourceType.Observation, Id('obs2'), null, null);
final search = await resourceDao.getResourceType(
'newPw',
resourceTypes: [R4ResourceType.Observation],
);
expect(search.length, 2);
final idList = [];
for (final obs in search) {
idList.add(obs.id.toString());
}
expect(idList.contains('obs1'), true);
expect(idList.contains('obs2'), false);
expect(idList.contains('obs3'), true);
});
test('Delete All Observations', () async {
await resourceDao.deleteSingleType('newPw',
resourceType: R4ResourceType.Observation);
final search = await resourceDao.getAll('newPw');
expect(search.length, 2);
final patList = search.toList();
final orgList = search.toList();
patList.retainWhere(
(resource) => resource.resourceType == R4ResourceType.Patient);
orgList.retainWhere(
(resource) => resource.resourceType == R4ResourceType.Organization);
expect(patList.length, 1);
expect(patList.length, 1);
});
test('Delete All Resources', () async {
await resourceDao.deleteAllResources('newPw');
final search = await resourceDao.getAll('newPw');
expect(search.length, 0);
await resourceDao.updatePw('newPw', null);
});
});
} Download Details:
Author: MayJuun
Source Code: https://github.com/MayJuun/fhir/tree/main/fhir_db
1648667160
LaravelS: Glue for using Swoole in Laravel Or Lumen
🚀 LaravelS is
an out-of-the-box adapterbetween Swoole and Laravel/Lumen.
Please Watch this repository to get the latest updates.
_ _ _____
| | | |/ ____|
| | __ _ _ __ __ ___ _____| | (___
| | / _` | '__/ _` \ \ / / _ \ |\___ \
| |___| (_| | | | (_| |\ V / __/ |____) |
|______\__,_|_| \__,_| \_/ \___|_|_____/ Features
Built-in Http/WebSocket server
Memory resident
Gracefully reload
Automatically reload after modifying code
Support Laravel/Lumen both, good compatibility
Simple & Out of the box
Benchmark
Which is the fastest web framework?
TechEmpower Framework Benchmarks
Requirements
| Dependency | Requirement |
|---|---|
| PHP | >= 5.5.9 Recommend PHP7+ |
| Swoole | >= 1.7.19 No longer support PHP5 since 2.0.12 Recommend 4.5.0+ |
| Laravel/Lumen | >= 5.1 Recommend 8.0+ |
Install
1.Require package via Composer(packagist).
composer require "hhxsv5/laravel-s:~3.7.0" -vvv
# Make sure that your composer.lock file is under the VCS2.Register service provider(pick one of two).
Laravel: in config/app.php file, Laravel 5.5+ supports package discovery automatically, you should skip this step
'providers' => [
//...
Hhxsv5\LaravelS\Illuminate\LaravelSServiceProvider::class,
],Lumen: in bootstrap/app.php file
$app->register(Hhxsv5\LaravelS\Illuminate\LaravelSServiceProvider::class);3.Publish configuration and binaries.
After upgrading LaravelS, you need to republish; click here to see the change notes of each version.
php artisan laravels publish
# Configuration: config/laravels.php
# Binary: bin/laravels bin/fswatch bin/inotify4.Change config/laravels.php: listen_ip, listen_port, refer Settings.
5.Performance tuning
Number of Workers: LaravelS uses Swoole's Synchronous IO mode, the larger the worker_num setting, the better the concurrency performance, but it will cause more memory usage and process switching overhead. If one request takes 100ms, in order to provide 1000QPS concurrency, at least 100 Worker processes need to be configured. The calculation method is: worker_num = 1000QPS/(1s/1ms) = 100, so incremental pressure testing is needed to calculate the best worker_num.
Run
Please read the notices carefully before running, Important notices(IMPORTANT).
- Commands:
php bin/laravels {start|stop|restart|reload|info|help}.
| Command | Description |
|---|---|
| start | Start LaravelS, list the processes by "ps -ef|grep laravels" |
| stop | Stop LaravelS, and trigger the method onStop of Custom process |
| restart | Restart LaravelS: Stop gracefully before starting; The service is unavailable until startup is complete |
| reload | Reload all Task/Worker/Timer processes which contain your business codes, and trigger the method onReload of Custom process, CANNOT reload Master/Manger processes. After modifying config/laravels.php, you only have to call restart to restart |
| info | Display component version information |
| help | Display help information |
- Boot options for the commands
startandrestart.
| Option | Description |
|---|---|
| -d|--daemonize | Run as a daemon, this option will override the swoole.daemonize setting in laravels.php |
| -e|--env | The environment the command should run under, such as --env=testing will use the configuration file .env.testing firstly, this feature requires Laravel 5.2+ |
| -i|--ignore | Ignore checking PID file of Master process |
| -x|--x-version | The version(branch) of the current project, stored in $_ENV/$_SERVER, access via $_ENV['X_VERSION'] $_SERVER['X_VERSION'] $request->server->get('X_VERSION') |
Runtimefiles:startwill automatically executephp artisan laravels configand generate these files, developers generally don't need to pay attention to them, it's recommended to add them to.gitignore.
| File | Description |
|---|---|
| storage/laravels.conf | LaravelS's runtime configuration file |
| storage/laravels.pid | PID file of Master process |
| storage/laravels-timer-process.pid | PID file of the Timer process |
| storage/laravels-custom-processes.pid | PID file of all custom processes |
Deploy
It is recommended to supervise the main process through Supervisord, the premise is without option
-dand to setswoole.daemonizetofalse.
[program:laravel-s-test]
directory=/var/www/laravel-s-test
command=/usr/local/bin/php bin/laravels start -i
numprocs=1
autostart=true
autorestart=true
startretries=3
user=www-data
redirect_stderr=true
stdout_logfile=/var/log/supervisor/%(program_name)s.log
Cooperate with Nginx (Recommended)
Demo.
gzip on;
gzip_min_length 1024;
gzip_comp_level 2;
gzip_types text/plain text/css text/javascript application/json application/javascript application/x-javascript application/xml application/x-httpd-php image/jpeg image/gif image/png font/ttf font/otf image/svg+xml;
gzip_vary on;
gzip_disable "msie6";
upstream swoole {
# Connect IP:Port
server 127.0.0.1:5200 weight=5 max_fails=3 fail_timeout=30s;
# Connect UnixSocket Stream file, tips: put the socket file in the /dev/shm directory to get better performance
#server unix:/yourpath/laravel-s-test/storage/laravels.sock weight=5 max_fails=3 fail_timeout=30s;
#server 192.168.1.1:5200 weight=3 max_fails=3 fail_timeout=30s;
#server 192.168.1.2:5200 backup;
keepalive 16;
}
server {
listen 80;
# Don't forget to bind the host
server_name laravels.com;
root /yourpath/laravel-s-test/public;
access_log /yourpath/log/nginx/$server_name.access.log main;
autoindex off;
index index.html index.htm;
# Nginx handles the static resources(recommend enabling gzip), LaravelS handles the dynamic resource.
location / {
try_files $uri @laravels;
}
# Response 404 directly when request the PHP file, to avoid exposing public/*.php
#location ~* \.php$ {
# return 404;
#}
location @laravels {
# proxy_connect_timeout 60s;
# proxy_send_timeout 60s;
# proxy_read_timeout 120s;
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Real-PORT $remote_port;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_set_header Scheme $scheme;
proxy_set_header Server-Protocol $server_protocol;
proxy_set_header Server-Name $server_name;
proxy_set_header Server-Addr $server_addr;
proxy_set_header Server-Port $server_port;
# "swoole" is the upstream
proxy_pass http://swoole;
}
}Cooperate with Apache
LoadModule proxy_module /yourpath/modules/mod_proxy.so
LoadModule proxy_balancer_module /yourpath/modules/mod_proxy_balancer.so
LoadModule lbmethod_byrequests_module /yourpath/modules/mod_lbmethod_byrequests.so
LoadModule proxy_http_module /yourpath/modules/mod_proxy_http.so
LoadModule slotmem_shm_module /yourpath/modules/mod_slotmem_shm.so
LoadModule rewrite_module /yourpath/modules/mod_rewrite.so
LoadModule remoteip_module /yourpath/modules/mod_remoteip.so
LoadModule deflate_module /yourpath/modules/mod_deflate.so
<IfModule deflate_module>
SetOutputFilter DEFLATE
DeflateCompressionLevel 2
AddOutputFilterByType DEFLATE text/html text/plain text/css text/javascript application/json application/javascript application/x-javascript application/xml application/x-httpd-php image/jpeg image/gif image/png font/ttf font/otf image/svg+xml
</IfModule>
<VirtualHost *:80>
# Don't forget to bind the host
ServerName www.laravels.com
ServerAdmin hhxsv5@sina.com
DocumentRoot /yourpath/laravel-s-test/public;
DirectoryIndex index.html index.htm
<Directory "/">
AllowOverride None
Require all granted
</Directory>
RemoteIPHeader X-Forwarded-For
ProxyRequests Off
ProxyPreserveHost On
<Proxy balancer://laravels>
BalancerMember http://192.168.1.1:5200 loadfactor=7
#BalancerMember http://192.168.1.2:5200 loadfactor=3
#BalancerMember http://192.168.1.3:5200 loadfactor=1 status=+H
ProxySet lbmethod=byrequests
</Proxy>
#ProxyPass / balancer://laravels/
#ProxyPassReverse / balancer://laravels/
# Apache handles the static resources, LaravelS handles the dynamic resource.
RewriteEngine On
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-d
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-f
RewriteRule ^/(.*)$ balancer://laravels%{REQUEST_URI} [P,L]
ErrorLog ${APACHE_LOG_DIR}/www.laravels.com.error.log
CustomLog ${APACHE_LOG_DIR}/www.laravels.com.access.log combined
</VirtualHost>Enable WebSocket server
The Listening address of WebSocket Sever is the same as Http Server.
1.Create WebSocket Handler class, and implement interface WebSocketHandlerInterface.The instant is automatically instantiated when start, you do not need to manually create it.
namespace App\Services;
use Hhxsv5\LaravelS\Swoole\WebSocketHandlerInterface;
use Swoole\Http\Request;
use Swoole\Http\Response;
use Swoole\WebSocket\Frame;
use Swoole\WebSocket\Server;
/**
* @see https://www.swoole.co.uk/docs/modules/swoole-websocket-server
*/
class WebSocketService implements WebSocketHandlerInterface
{
// Declare constructor without parameters
public function __construct()
{
}
// public function onHandShake(Request $request, Response $response)
// {
// Custom handshake: https://www.swoole.co.uk/docs/modules/swoole-websocket-server-on-handshake
// The onOpen event will be triggered automatically after a successful handshake
// }
public function onOpen(Server $server, Request $request)
{
// Before the onOpen event is triggered, the HTTP request to establish the WebSocket has passed the Laravel route,
// so Laravel's Request, Auth information are readable, Session is readable and writable, but only in the onOpen event.
// \Log::info('New WebSocket connection', [$request->fd, request()->all(), session()->getId(), session('xxx'), session(['yyy' => time()])]);
// The exceptions thrown here will be caught by the upper layer and recorded in the Swoole log. Developers need to try/catch manually.
$server->push($request->fd, 'Welcome to LaravelS');
}
public function onMessage(Server $server, Frame $frame)
{
// \Log::info('Received message', [$frame->fd, $frame->data, $frame->opcode, $frame->finish]);
// The exceptions thrown here will be caught by the upper layer and recorded in the Swoole log. Developers need to try/catch manually.
$server->push($frame->fd, date('Y-m-d H:i:s'));
}
public function onClose(Server $server, $fd, $reactorId)
{
// The exceptions thrown here will be caught by the upper layer and recorded in the Swoole log. Developers need to try/catch manually.
}
}2.Modify config/laravels.php.
// ...
'websocket' => [
'enable' => true, // Note: set enable to true
'handler' => \App\Services\WebSocketService::class,
],
'swoole' => [
//...
// Must set dispatch_mode in (2, 4, 5), see https://www.swoole.co.uk/docs/modules/swoole-server/configuration
'dispatch_mode' => 2,
//...
],
// ...3.Use SwooleTable to bind FD & UserId, optional, Swoole Table Demo. Also you can use the other global storage services, like Redis/Memcached/MySQL, but be careful that FD will be possible conflicting between multiple Swoole Servers.
4.Cooperate with Nginx (Recommended)
Refer WebSocket Proxy
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
upstream swoole {
# Connect IP:Port
server 127.0.0.1:5200 weight=5 max_fails=3 fail_timeout=30s;
# Connect UnixSocket Stream file, tips: put the socket file in the /dev/shm directory to get better performance
#server unix:/yourpath/laravel-s-test/storage/laravels.sock weight=5 max_fails=3 fail_timeout=30s;
#server 192.168.1.1:5200 weight=3 max_fails=3 fail_timeout=30s;
#server 192.168.1.2:5200 backup;
keepalive 16;
}
server {
listen 80;
# Don't forget to bind the host
server_name laravels.com;
root /yourpath/laravel-s-test/public;
access_log /yourpath/log/nginx/$server_name.access.log main;
autoindex off;
index index.html index.htm;
# Nginx handles the static resources(recommend enabling gzip), LaravelS handles the dynamic resource.
location / {
try_files $uri @laravels;
}
# Response 404 directly when request the PHP file, to avoid exposing public/*.php
#location ~* \.php$ {
# return 404;
#}
# Http and WebSocket are concomitant, Nginx identifies them by "location"
# !!! The location of WebSocket is "/ws"
# Javascript: var ws = new WebSocket("ws://laravels.com/ws");
location =/ws {
# proxy_connect_timeout 60s;
# proxy_send_timeout 60s;
# proxy_read_timeout: Nginx will close the connection if the proxied server does not send data to Nginx in 60 seconds; At the same time, this close behavior is also affected by heartbeat setting of Swoole.
# proxy_read_timeout 60s;
proxy_http_version 1.1;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Real-PORT $remote_port;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_set_header Scheme $scheme;
proxy_set_header Server-Protocol $server_protocol;
proxy_set_header Server-Name $server_name;
proxy_set_header Server-Addr $server_addr;
proxy_set_header Server-Port $server_port;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_pass http://swoole;
}
location @laravels {
# proxy_connect_timeout 60s;
# proxy_send_timeout 60s;
# proxy_read_timeout 60s;
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Real-PORT $remote_port;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_set_header Scheme $scheme;
proxy_set_header Server-Protocol $server_protocol;
proxy_set_header Server-Name $server_name;
proxy_set_header Server-Addr $server_addr;
proxy_set_header Server-Port $server_port;
proxy_pass http://swoole;
}
}5.Heartbeat setting
Heartbeat setting of Swoole
// config/laravels.php
'swoole' => [
//...
// All connections are traversed every 60 seconds. If a connection does not send any data to the server within 600 seconds, the connection will be forced to close.
'heartbeat_idle_time' => 600,
'heartbeat_check_interval' => 60,
//...
],Proxy read timeout of Nginx
# Nginx will close the connection if the proxied server does not send data to Nginx in 60 seconds
proxy_read_timeout 60s;6.Push data in controller
namespace App\Http\Controllers;
class TestController extends Controller
{
public function push()
{
$fd = 1; // Find fd by userId from a map [userId=>fd].
/**@var \Swoole\WebSocket\Server $swoole */
$swoole = app('swoole');
$success = $swoole->push($fd, 'Push data to fd#1 in Controller');
var_dump($success);
}
}Listen events
System events
Usually, you can reset/destroy some
global/staticvariables, or change the currentRequest/Responseobject.
laravels.received_request After LaravelS parsed Swoole\Http\Request to Illuminate\Http\Request, before Laravel's Kernel handles this request.
// Edit file `app/Providers/EventServiceProvider.php`, add the following code into method `boot`
// If no variable $events, you can also call Facade \Event::listen().
$events->listen('laravels.received_request', function (\Illuminate\Http\Request $req, $app) {
$req->query->set('get_key', 'hhxsv5');// Change query of request
$req->request->set('post_key', 'hhxsv5'); // Change post of request
});laravels.generated_response After Laravel's Kernel handled the request, before LaravelS parses Illuminate\Http\Response to Swoole\Http\Response.
// Edit file `app/Providers/EventServiceProvider.php`, add the following code into method `boot`
// If no variable $events, you can also call Facade \Event::listen().
$events->listen('laravels.generated_response', function (\Illuminate\Http\Request $req, \Symfony\Component\HttpFoundation\Response $rsp, $app) {
$rsp->headers->set('header-key', 'hhxsv5');// Change header of response
});Customized asynchronous events
This feature depends on
AsyncTaskofSwoole, your need to setswoole.task_worker_numinconfig/laravels.phpfirstly. The performance of asynchronous event processing is influenced by number of Swoole task process, you need to set task_worker_num appropriately.
1.Create event class.
use Hhxsv5\LaravelS\Swoole\Task\Event;
class TestEvent extends Event
{
protected $listeners = [
// Listener list
TestListener1::class,
// TestListener2::class,
];
private $data;
public function __construct($data)
{
$this->data = $data;
}
public function getData()
{
return $this->data;
}
}2.Create listener class.
use Hhxsv5\LaravelS\Swoole\Task\Task;
use Hhxsv5\LaravelS\Swoole\Task\Listener;
class TestListener1 extends Listener
{
/**
* @var TestEvent
*/
protected $event;
public function handle()
{
\Log::info(__CLASS__ . ':handle start', [$this->event->getData()]);
sleep(2);// Simulate the slow codes
// Deliver task in CronJob, but NOT support callback finish() of task.
// Note: Modify task_ipc_mode to 1 or 2 in config/laravels.php, see https://www.swoole.co.uk/docs/modules/swoole-server/configuration
$ret = Task::deliver(new TestTask('task data'));
var_dump($ret);
// The exceptions thrown here will be caught by the upper layer and recorded in the Swoole log. Developers need to try/catch manually.
}
}3.Fire event.
// Create instance of event and fire it, "fire" is asynchronous.
use Hhxsv5\LaravelS\Swoole\Task\Event;
$event = new TestEvent('event data');
// $event->delay(10); // Delay 10 seconds to fire event
// $event->setTries(3); // When an error occurs, try 3 times in total
$success = Event::fire($event);
var_dump($success);// Return true if sucess, otherwise falseAsynchronous task queue
This feature depends on
AsyncTaskofSwoole, your need to setswoole.task_worker_numinconfig/laravels.phpfirstly. The performance of task processing is influenced by number of Swoole task process, you need to set task_worker_num appropriately.
1.Create task class.
use Hhxsv5\LaravelS\Swoole\Task\Task;
class TestTask extends Task
{
private $data;
private $result;
public function __construct($data)
{
$this->data = $data;
}
// The logic of task handling, run in task process, CAN NOT deliver task
public function handle()
{
\Log::info(__CLASS__ . ':handle start', [$this->data]);
sleep(2);// Simulate the slow codes
// The exceptions thrown here will be caught by the upper layer and recorded in the Swoole log. Developers need to try/catch manually.
$this->result = 'the result of ' . $this->data;
}
// Optional, finish event, the logic of after task handling, run in worker process, CAN deliver task
public function finish()
{
\Log::info(__CLASS__ . ':finish start', [$this->result]);
Task::deliver(new TestTask2('task2 data')); // Deliver the other task
}
}2.Deliver task.
// Create instance of TestTask and deliver it, "deliver" is asynchronous.
use Hhxsv5\LaravelS\Swoole\Task\Task;
$task = new TestTask('task data');
// $task->delay(3);// delay 3 seconds to deliver task
// $task->setTries(3); // When an error occurs, try 3 times in total
$ret = Task::deliver($task);
var_dump($ret);// Return true if sucess, otherwise falseMillisecond cron job
Wrapper cron job base on Swoole's Millisecond Timer, replace
LinuxCrontab.
1.Create cron job class.
namespace App\Jobs\Timer;
use App\Tasks\TestTask;
use Swoole\Coroutine;
use Hhxsv5\LaravelS\Swoole\Task\Task;
use Hhxsv5\LaravelS\Swoole\Timer\CronJob;
class TestCronJob extends CronJob
{
protected $i = 0;
// !!! The `interval` and `isImmediate` of cron job can be configured in two ways(pick one of two): one is to overload the corresponding method, and the other is to pass parameters when registering cron job.
// --- Override the corresponding method to return the configuration: begin
public function interval()
{
return 1000;// Run every 1000ms
}
public function isImmediate()
{
return false;// Whether to trigger `run` immediately after setting up
}
// --- Override the corresponding method to return the configuration: end
public function run()
{
\Log::info(__METHOD__, ['start', $this->i, microtime(true)]);
// do something
// sleep(1); // Swoole < 2.1
Coroutine::sleep(1); // Swoole>=2.1 Coroutine will be automatically created for run().
$this->i++;
\Log::info(__METHOD__, ['end', $this->i, microtime(true)]);
if ($this->i >= 10) { // Run 10 times only
\Log::info(__METHOD__, ['stop', $this->i, microtime(true)]);
$this->stop(); // Stop this cron job, but it will run again after restart/reload.
// Deliver task in CronJob, but NOT support callback finish() of task.
// Note: Modify task_ipc_mode to 1 or 2 in config/laravels.php, see https://www.swoole.co.uk/docs/modules/swoole-server/configuration
$ret = Task::deliver(new TestTask('task data'));
var_dump($ret);
}
// The exceptions thrown here will be caught by the upper layer and recorded in the Swoole log. Developers need to try/catch manually.
}
}2.Register cron job.
// Register cron jobs in file "config/laravels.php"
[
// ...
'timer' => [
'enable' => true, // Enable Timer
'jobs' => [ // The list of cron job
// Enable LaravelScheduleJob to run `php artisan schedule:run` every 1 minute, replace Linux Crontab
// \Hhxsv5\LaravelS\Illuminate\LaravelScheduleJob::class,
// Two ways to configure parameters:
// [\App\Jobs\Timer\TestCronJob::class, [1000, true]], // Pass in parameters when registering
\App\Jobs\Timer\TestCronJob::class, // Override the corresponding method to return the configuration
],
'max_wait_time' => 5, // Max waiting time of reloading
// Enable the global lock to ensure that only one instance starts the timer when deploying multiple instances. This feature depends on Redis, please see https://laravel.com/docs/7.x/redis
'global_lock' => false,
'global_lock_key' => config('app.name', 'Laravel'),
],
// ...
];3.Note: it will launch multiple timers when build the server cluster, so you need to make sure that launch one timer only to avoid running repetitive task.
4.LaravelS v3.4.0 starts to support the hot restart [Reload] Timer process. After LaravelS receives the SIGUSR1 signal, it waits for max_wait_time(default 5) seconds to end the process, then the Manager process will pull up the Timer process again.
5.If you only need to use minute-level scheduled tasks, it is recommended to enable Hhxsv5\LaravelS\Illuminate\LaravelScheduleJob instead of Linux Crontab, so that you can follow the coding habits of Laravel task scheduling and configure Kernel.
// app/Console/Kernel.php
protected function schedule(Schedule $schedule)
{
// runInBackground() will start a new child process to execute the task. This is asynchronous and will not affect the execution timing of other tasks.
$schedule->command(TestCommand::class)->runInBackground()->everyMinute();
}Automatically reload after modifying code
Via inotify, support Linux only.
1.Install inotify extension.
2.Turn on the switch in Settings.
3.Notice: Modify the file only in Linux to receive the file change events. It's recommended to use the latest Docker. Vagrant Solution.
Via fswatch, support OS X/Linux/Windows.
1.Install fswatch.
2.Run command in your project root directory.
# Watch current directory
./bin/fswatch
# Watch app directory
./bin/fswatch ./appVia inotifywait, support Linux.
1.Install inotify-tools.
2.Run command in your project root directory.
# Watch current directory
./bin/inotify
# Watch app directory
./bin/inotify ./appWhen the above methods does not work, the ultimate solution: set max_request=1,worker_num=1, so that Worker process will restart after processing a request. The performance of this method is very poor, so only development environment use.
Get the instance of SwooleServer in your project
/**
* $swoole is the instance of `Swoole\WebSocket\Server` if enable WebSocket server, otherwise `Swoole\Http\Server`
* @var \Swoole\WebSocket\Server|\Swoole\Http\Server $swoole
*/
$swoole = app('swoole');
var_dump($swoole->stats());
$swoole->push($fd, 'Push WebSocket message');Use SwooleTable
1.Define Table, support multiple.
All defined tables will be created before Swoole starting.
// in file "config/laravels.php"
[
// ...
'swoole_tables' => [
// Scene：bind UserId & FD in WebSocket
'ws' => [// The Key is table name, will add suffix "Table" to avoid naming conflicts. Here defined a table named "wsTable"
'size' => 102400,// The max size
'column' => [// Define the columns
['name' => 'value', 'type' => \Swoole\Table::TYPE_INT, 'size' => 8],
],
],
//...Define the other tables
],
// ...
];2.Access Table: all table instances will be bound on SwooleServer, access by app('swoole')->xxxTable.
namespace App\Services;
use Hhxsv5\LaravelS\Swoole\WebSocketHandlerInterface;
use Swoole\Http\Request;
use Swoole\WebSocket\Frame;
use Swoole\WebSocket\Server;
class WebSocketService implements WebSocketHandlerInterface
{
/**@var \Swoole\Table $wsTable */
private $wsTable;
public function __construct()
{
$this->wsTable = app('swoole')->wsTable;
}
// Scene：bind UserId & FD in WebSocket
public function onOpen(Server $server, Request $request)
{
// var_dump(app('swoole') === $server);// The same instance
/**
* Get the currently logged in user
* This feature requires that the path to establish a WebSocket connection go through middleware such as Authenticate.
* E.g:
* Browser side: var ws = new WebSocket("ws://127.0.0.1:5200/ws");
* Then the /ws route in Laravel needs to add the middleware like Authenticate.
* Route::get('/ws', function () {
* // Respond any content with status code 200
* return 'websocket';
* })->middleware(['auth']);
*/
// $user = Auth::user();
// $userId = $user ? $user->id : 0; // 0 means a guest user who is not logged in
$userId = mt_rand(1000, 10000);
// if (!$userId) {
// // Disconnect the connections of unlogged users
// $server->disconnect($request->fd);
// return;
// }
$this->wsTable->set('uid:' . $userId, ['value' => $request->fd]);// Bind map uid to fd
$this->wsTable->set('fd:' . $request->fd, ['value' => $userId]);// Bind map fd to uid
$server->push($request->fd, "Welcome to LaravelS #{$request->fd}");
}
public function onMessage(Server $server, Frame $frame)
{
// Broadcast
foreach ($this->wsTable as $key => $row) {
if (strpos($key, 'uid:') === 0 && $server->isEstablished($row['value'])) {
$content = sprintf('Broadcast: new message "%s" from #%d', $frame->data, $frame->fd);
$server->push($row['value'], $content);
}
}
}
public function onClose(Server $server, $fd, $reactorId)
{
$uid = $this->wsTable->get('fd:' . $fd);
if ($uid !== false) {
$this->wsTable->del('uid:' . $uid['value']); // Unbind uid map
}
$this->wsTable->del('fd:' . $fd);// Unbind fd map
$server->push($fd, "Goodbye #{$fd}");
}
}Multi-port mixed protocol
For more information, please refer to Swoole Server AddListener
To make our main server support more protocols not just Http and WebSocket, we bring the feature multi-port mixed protocol of Swoole in LaravelS and name it Socket. Now, you can build TCP/UDP applications easily on top of Laravel.
Create Socket handler class, and extend Hhxsv5\LaravelS\Swoole\Socket\{TcpSocket|UdpSocket|Http|WebSocket}.
namespace App\Sockets;
use Hhxsv5\LaravelS\Swoole\Socket\TcpSocket;
use Swoole\Server;
class TestTcpSocket extends TcpSocket
{
public function onConnect(Server $server, $fd, $reactorId)
{
\Log::info('New TCP connection', [$fd]);
$server->send($fd, 'Welcome to LaravelS.');
}
public function onReceive(Server $server, $fd, $reactorId, $data)
{
\Log::info('Received data', [$fd, $data]);
$server->send($fd, 'LaravelS: ' . $data);
if ($data === "quit\r\n") {
$server->send($fd, 'LaravelS: bye' . PHP_EOL);
$server->close($fd);
}
}
public function onClose(Server $server, $fd, $reactorId)
{
\Log::info('Close TCP connection', [$fd]);
$server->send($fd, 'Goodbye');
}
}These Socket connections share the same worker processes with your HTTP/WebSocket connections. So it won't be a problem at all if you want to deliver tasks, use SwooleTable, even Laravel components such as DB, Eloquent and so on. At the same time, you can access Swoole\Server\Port object directly by member property swoolePort.
public function onReceive(Server $server, $fd, $reactorId, $data)
{
$port = $this->swoolePort; // Get the `Swoole\Server\Port` object
}namespace App\Http\Controllers;
class TestController extends Controller
{
public function test()
{
/**@var \Swoole\Http\Server|\Swoole\WebSocket\Server $swoole */
$swoole = app('swoole');
// $swoole->ports: Traverse all Port objects, https://www.swoole.co.uk/docs/modules/swoole-server/multiple-ports
$port = $swoole->ports[0]; // Get the `Swoole\Server\Port` object, $port[0] is the port of the main server
foreach ($port->connections as $fd) { // Traverse all connections
// $swoole->send($fd, 'Send tcp message');
// if($swoole->isEstablished($fd)) {
// $swoole->push($fd, 'Send websocket message');
// }
}
}
}Register Sockets.
// Edit `config/laravels.php`
//...
'sockets' => [
[
'host' => '127.0.0.1',
'port' => 5291,
'type' => SWOOLE_SOCK_TCP,// Socket type: SWOOLE_SOCK_TCP/SWOOLE_SOCK_TCP6/SWOOLE_SOCK_UDP/SWOOLE_SOCK_UDP6/SWOOLE_UNIX_DGRAM/SWOOLE_UNIX_STREAM
'settings' => [// Swoole settings：https://www.swoole.co.uk/docs/modules/swoole-server-methods#swoole_server-addlistener
'open_eof_check' => true,
'package_eof' => "\r\n",
],
'handler' => \App\Sockets\TestTcpSocket::class,
'enable' => true, // whether to enable, default true
],
],About the heartbeat configuration, it can only be set on the main server and cannot be configured on Socket, but the Socket inherits the heartbeat configuration of the main server.
For TCP socket, onConnect and onClose events will be blocked when dispatch_mode of Swoole is 1/3, so if you want to unblock these two events please set dispatch_mode to 2/4/5.
'swoole' => [
//...
'dispatch_mode' => 2,
//...
];Test.
TCP: telnet 127.0.0.1 5291
UDP: [Linux] echo "Hello LaravelS" > /dev/udp/127.0.0.1/5292
Register example of other protocols.
- UDP
- Http
- WebSocket: The main server must
turn on WebSocket, that is, setwebsocket.enabletotrue.
Coroutine
Warning: The order of code execution in the coroutine is out of order. The data of the request level should be isolated by the coroutine ID. However, there are many singleton and static attributes in Laravel/Lumen, the data between different requests will affect each other, it's Unsafe. For example, the database connection is a singleton, the same database connection shares the same PDO resource. This is fine in the synchronous blocking mode, but it does not work in the asynchronous coroutine mode. Each query needs to create different connections and maintain IO state of different connections, which requires a connection pool.
DO NOT enable the coroutine, only the custom process can use the coroutine.
Custom process
Support developers to create special work processes for monitoring, reporting, or other special tasks. Refer addProcess.
Create Proccess class, implements CustomProcessInterface.
namespace App\Processes;
use App\Tasks\TestTask;
use Hhxsv5\LaravelS\Swoole\Process\CustomProcessInterface;
use Hhxsv5\LaravelS\Swoole\Task\Task;
use Swoole\Coroutine;
use Swoole\Http\Server;
use Swoole\Process;
class TestProcess implements CustomProcessInterface
{
/**
* @var bool Quit tag for Reload updates
*/
private static $quit = false;
public static function callback(Server $swoole, Process $process)
{
// The callback method cannot exit. Once exited, Manager process will automatically create the process
while (!self::$quit) {
\Log::info('Test process: running');
// sleep(1); // Swoole < 2.1
Coroutine::sleep(1); // Swoole>=2.1: Coroutine & Runtime will be automatically enabled for callback().
// Deliver task in custom process, but NOT support callback finish() of task.
// Note: Modify task_ipc_mode to 1 or 2 in config/laravels.php, see https://www.swoole.co.uk/docs/modules/swoole-server/configuration
$ret = Task::deliver(new TestTask('task data'));
var_dump($ret);
// The upper layer will catch the exception thrown in the callback and record it in the Swoole log, and then this process will exit. The Manager process will re-create the process after 3 seconds, so developers need to try/catch to catch the exception by themselves to avoid frequent process creation.
// throw new \Exception('an exception');
}
}
// Requirements: LaravelS >= v3.4.0 & callback() must be async non-blocking program.
public static function onReload(Server $swoole, Process $process)
{
// Stop the process...
// Then end process
\Log::info('Test process: reloading');
self::$quit = true;
// $process->exit(0); // Force exit process
}
// Requirements: LaravelS >= v3.7.4 & callback() must be async non-blocking program.
public static function onStop(Server $swoole, Process $process)
{
// Stop the process...
// Then end process
\Log::info('Test process: stopping');
self::$quit = true;
// $process->exit(0); // Force exit process
}
}Register TestProcess.
// Edit `config/laravels.php`
// ...
'processes' => [
'test' => [ // Key name is process name
'class' => \App\Processes\TestProcess::class,
'redirect' => false, // Whether redirect stdin/stdout, true or false
'pipe' => 0, // The type of pipeline, 0: no pipeline 1: SOCK_STREAM 2: SOCK_DGRAM
'enable' => true, // Whether to enable, default true
//'num' => 3 // To create multiple processes of this class, default is 1
//'queue' => [ // Enable message queue as inter-process communication, configure empty array means use default parameters
// 'msg_key' => 0, // The key of the message queue. Default: ftok(__FILE__, 1).
// 'mode' => 2, // Communication mode, default is 2, which means contention mode
// 'capacity' => 8192, // The length of a single message, is limited by the operating system kernel parameters. The default is 8192, and the maximum is 65536
//],
//'restart_interval' => 5, // After the process exits abnormally, how many seconds to wait before restarting the process, default 5 seconds
],
],Note: The callback() cannot quit. If quit, the Manager process will re-create the process.
Example: Write data to a custom process.
// config/laravels.php
'processes' => [
'test' => [
'class' => \App\Processes\TestProcess::class,
'redirect' => false,
'pipe' => 1,
],
],// app/Processes/TestProcess.php
public static function callback(Server $swoole, Process $process)
{
while ($data = $process->read()) {
\Log::info('TestProcess: read data', [$data]);
$process->write('TestProcess: ' . $data);
}
}// app/Http/Controllers/TestController.php
public function testProcessWrite()
{
/**@var \Swoole\Process $process */
$process = app('swoole')->customProcesses['test'];
$process->write('TestController: write data' . time());
var_dump($process->read());
}Common components
Apollo
LaravelSwill pull theApolloconfiguration and write it to the.envfile when starting. At the same time,LaravelSwill start the custom processapolloto monitor the configuration and automaticallyreloadwhen the configuration changes.
Enable Apollo: add --enable-apollo and Apollo parameters to the startup parameters.
php bin/laravels start --enable-apollo --apollo-server=http://127.0.0.1:8080 --apollo-app-id=LARAVEL-S-TESTSupport hot updates(optional).
// Edit `config/laravels.php`
'processes' => Hhxsv5\LaravelS\Components\Apollo\Process::getDefinition(),// When there are other custom process configurations
'processes' => [
'test' => [
'class' => \App\Processes\TestProcess::class,
'redirect' => false,
'pipe' => 1,
],
// ...
] + Hhxsv5\LaravelS\Components\Apollo\Process::getDefinition(),List of available parameters.
| Parameter | Description | Default | Demo |
|---|---|---|---|
| apollo-server | Apollo server URL | - | --apollo-server=http://127.0.0.1:8080 |
| apollo-app-id | Apollo APP ID | - | --apollo-app-id=LARAVEL-S-TEST |
| apollo-namespaces | The namespace to which the APP belongs, support specify the multiple | application | --apollo-namespaces=application --apollo-namespaces=env |
| apollo-cluster | The cluster to which the APP belongs | default | --apollo-cluster=default |
| apollo-client-ip | IP of current instance, can also be used for grayscale publishing | Local intranet IP | --apollo-client-ip=10.2.1.83 |
| apollo-pull-timeout | Timeout time(seconds) when pulling configuration | 5 | --apollo-pull-timeout=5 |
| apollo-backup-old-env | Whether to backup the old configuration file when updating the configuration file .env | false | --apollo-backup-old-env |
Prometheus
Support Prometheus monitoring and alarm, Grafana visually view monitoring metrics. Please refer to Docker Compose for the environment construction of Prometheus and Grafana.
Require extension APCu >= 5.0.0, please install it by pecl install apcu.
Copy the configuration file prometheus.php to the config directory of your project. Modify the configuration as appropriate.
# Execute commands in the project root directory
cp vendor/hhxsv5/laravel-s/config/prometheus.php config/If your project is Lumen, you also need to manually load the configuration $app->configure('prometheus'); in bootstrap/app.php.
Configure global middleware: Hhxsv5\LaravelS\Components\Prometheus\RequestMiddleware::class. In order to count the request time consumption as accurately as possible, RequestMiddleware must be the first global middleware, which needs to be placed in front of other middleware.
Register ServiceProvider: Hhxsv5\LaravelS\Components\Prometheus\ServiceProvider::class.
Configure the CollectorProcess in config/laravels.php to collect the metrics of Swoole Worker/Task/Timer processes regularly.
'processes' => Hhxsv5\LaravelS\Components\Prometheus\CollectorProcess::getDefinition(),Create the route to output metrics.
use Hhxsv5\LaravelS\Components\Prometheus\Exporter;
Route::get('/actuator/prometheus', function () {
$result = app(Exporter::class)->render();
return response($result, 200, ['Content-Type' => Exporter::REDNER_MIME_TYPE]);
});Complete the configuration of Prometheus and start it.
global:
scrape_interval: 5s
scrape_timeout: 5s
evaluation_interval: 30s
scrape_configs:
- job_name: laravel-s-test
honor_timestamps: true
metrics_path: /actuator/prometheus
scheme: http
follow_redirects: true
static_configs:
- targets:
- 127.0.0.1:5200 # The ip and port of the monitored service
# Dynamically discovered using one of the supported service-discovery mechanisms
# https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config
# - job_name: laravels-eureka
# honor_timestamps: true
# scrape_interval: 5s
# metrics_path: /actuator/prometheus
# scheme: http
# follow_redirects: true
# eureka_sd_configs:
# - server: http://127.0.0.1:8080/eureka
# follow_redirects: true
# refresh_interval: 5sStart Grafana, then import panel json.
Other features
Configure Swoole events
Supported events:
| Event | Interface | When happened |
|---|---|---|
| ServerStart | Hhxsv5\LaravelS\Swoole\Events\ServerStartInterface | Occurs when the Master process is starting, this event should not handle complex business logic, and can only do some simple work of initialization. |
| ServerStop | Hhxsv5\LaravelS\Swoole\Events\ServerStopInterface | Occurs when the server exits normally, CANNOT use async or coroutine related APIs in this event. |
| WorkerStart | Hhxsv5\LaravelS\Swoole\Events\WorkerStartInterface | Occurs after the Worker/Task process is started, and the Laravel initialization has been completed. |
| WorkerStop | Hhxsv5\LaravelS\Swoole\Events\WorkerStopInterface | Occurs after the Worker/Task process exits normally |
| WorkerError | Hhxsv5\LaravelS\Swoole\Events\WorkerErrorInterface | Occurs when an exception or fatal error occurs in the Worker/Task process |
1.Create an event class to implement the corresponding interface.
namespace App\Events;
use Hhxsv5\LaravelS\Swoole\Events\ServerStartInterface;
use Swoole\Atomic;
use Swoole\Http\Server;
class ServerStartEvent implements ServerStartInterface
{
public function __construct()
{
}
public function handle(Server $server)
{
// Initialize a global counter (available across processes)
$server->atomicCount = new Atomic(2233);
// Invoked in controller: app('swoole')->atomicCount->get();
}
}namespace App\Events;
use Hhxsv5\LaravelS\Swoole\Events\WorkerStartInterface;
use Swoole\Http\Server;
class WorkerStartEvent implements WorkerStartInterface
{
public function __construct()
{
}
public function handle(Server $server, $workerId)
{
// Initialize a database connection pool
// DatabaseConnectionPool::init();
}
}2.Configuration.
// Edit `config/laravels.php`
'event_handlers' => [
'ServerStart' => [\App\Events\ServerStartEvent::class], // Trigger events in array order
'WorkerStart' => [\App\Events\WorkerStartEvent::class],
],Serverless
Alibaba Cloud Function Compute
1.Modify bootstrap/app.php and set the storage directory. Because the project directory is read-only, the /tmp directory can only be read and written.
$app->useStoragePath(env('APP_STORAGE_PATH', '/tmp/storage'));2.Create a shell script laravels_bootstrap and grant executable permission.
#!/usr/bin/env bash
set +e
# Create storage-related directories
mkdir -p /tmp/storage/app/public
mkdir -p /tmp/storage/framework/cache
mkdir -p /tmp/storage/framework/sessions
mkdir -p /tmp/storage/framework/testing
mkdir -p /tmp/storage/framework/views
mkdir -p /tmp/storage/logs
# Set the environment variable APP_STORAGE_PATH, please make sure it's the same as APP_STORAGE_PATH in .env
export APP_STORAGE_PATH=/tmp/storage
# Start LaravelS
php bin/laravels start3.Configure template.xml.
ROSTemplateFormatVersion: '2015-09-01'
Transform: 'Aliyun::Serverless-2018-04-03'
Resources:
laravel-s-demo:
Type: 'Aliyun::Serverless::Service'
Properties:
Description: 'LaravelS Demo for Serverless'
fc-laravel-s:
Type: 'Aliyun::Serverless::Function'
Properties:
Handler: laravels.handler
Runtime: custom
MemorySize: 512
Timeout: 30
CodeUri: ./
InstanceConcurrency: 10
EnvironmentVariables:
BOOTSTRAP_FILE: laravels_bootstrap
Important notices
Singleton Issue
Under FPM mode, singleton instances will be instantiated and recycled in every request, request start=>instantiate instance=>request end=>recycled instance.
Under Swoole Server, All singleton instances will be held in memory, different lifetime from FPM, request start=>instantiate instance=>request end=>do not recycle singleton instance. So need developer to maintain status of singleton instances in every request.
Common solutions:
Write a XxxCleaner class to clean up the singleton object state. This class implements the interface Hhxsv5\LaravelS\Illuminate\Cleaners\CleanerInterface and then registers it in cleaners of laravels.php.
Reset status of singleton instances by Middleware.
Re-register ServiceProvider, add XxxServiceProvider into register_providers of file laravels.php. So that reinitialize singleton instances in every request Refer.
Cleaners
Known issues
Known issues: a package of known issues and solutions.
Debugging method
Logging; if you want to output to the console, you can use stderr, Log::channel('stderr')->debug('debug message').
Laravel Dump Server(Laravel 5.7 has been integrated by default).
Read request
Read request by Illuminate\Http\Request Object, $_ENV is readable, $_SERVER is partially readable, CANNOT USE $_GET/$_POST/$_FILES/$_COOKIE/$_REQUEST/$_SESSION/$GLOBALS.
public function form(\Illuminate\Http\Request $request)
{
$name = $request->input('name');
$all = $request->all();
$sessionId = $request->cookie('sessionId');
$photo = $request->file('photo');
// Call getContent() to get the raw POST body, instead of file_get_contents('php://input')
$rawContent = $request->getContent();
//...
}Output response
Respond by Illuminate\Http\Response Object, compatible with echo/vardump()/print_r()，CANNOT USE functions dd()/exit()/die()/header()/setcookie()/http_response_code().
public function json()
{
return response()->json(['time' => time()])->header('header1', 'value1')->withCookie('c1', 'v1');
}Persistent connection
Singleton connection will be resident in memory, it is recommended to turn on persistent connection for better performance.
- Database connection, it
willreconnect automaticallyimmediatelyafter disconnect.
// config/database.php
'connections' => [
'my_conn' => [
'driver' => 'mysql',
'host' => env('DB_MY_CONN_HOST', 'localhost'),
'port' => env('DB_MY_CONN_PORT', 3306),
'database' => env('DB_MY_CONN_DATABASE', 'forge'),
'username' => env('DB_MY_CONN_USERNAME', 'forge'),
'password' => env('DB_MY_CONN_PASSWORD', ''),
'charset' => 'utf8mb4',
'collation' => 'utf8mb4_unicode_ci',
'prefix' => '',
'strict' => false,
'options' => [
// Enable persistent connection
\PDO::ATTR_PERSISTENT => true,
],
],
],- Redis connection, it
won'treconnect automaticallyimmediatelyafter disconnect, and will throw an exception about lost connection, reconnect next time. You need to make sure thatSELECT DBcorrectly before operating Redis every time.
// config/database.php
'redis' => [
'client' => env('REDIS_CLIENT', 'phpredis'), // It is recommended to use phpredis for better performance.
'default' => [
'host' => env('REDIS_HOST', 'localhost'),
'password' => env('REDIS_PASSWORD', null),
'port' => env('REDIS_PORT', 6379),
'database' => 0,
'persistent' => true, // Enable persistent connection
],
],About memory leaks
Avoid using global variables. If necessary, please clean or reset them manually.
Infinitely appending element into static/global variable will lead to OOM(Out of Memory).
class Test
{
public static $array = [];
public static $string = '';
}
// Controller
public function test(Request $req)
{
// Out of Memory
Test::$array[] = $req->input('param1');
Test::$string .= $req->input('param2');
}Memory leak detection method
Modify config/laravels.php: worker_num=1, max_request=1000000, remember to change it back after test;
Add routing /debug-memory-leak without route middleware to observe the memory changes of the Worker process;
Start LaravelS and request /debug-memory-leak until diff_mem is less than or equal to zero; if diff_mem is always greater than zero, it means that there may be a memory leak in Global Middleware or Laravel Framework;
After completing Step 3, alternately request the business routes and /debug-memory-leak (It is recommended to use ab/wrk to make a large number of requests for business routes), the initial increase in memory is normal. After a large number of requests for the business routes, if diff_mem is always greater than zero and curr_mem continues to increase, there is a high probability of memory leak; If curr_mem always changes within a certain range and does not continue to increase, there is a low probability of memory leak.
If you still can't solve it, max_request is the last guarantee.
Linux kernel parameter adjustment
Pressure test
Alternatives
Sponsor
Author: hhxsv5
Source Code: https://github.com/hhxsv5/laravel-s
License: MIT License
1649214000
LaravelS: Glue for using Swoole in Laravel Or Lumen.
🚀 LaravelS is
an out-of-the-box adapterbetween Swoole and Laravel/Lumen.
Please Watch this repository to get the latest updates.
Table of Contents
- Features
- Benchmark
- Requirements
- Install
- Run
- Deploy
- Cooperate with Nginx (Recommended)
- Cooperate with Apache
- Enable WebSocket server
- Listen events
- Asynchronous task queue
- Millisecond cron job
- Automatically reload after modifying code
- Get the instance of SwooleServer in your project
- Use SwooleTable
- Multi-port mixed protocol
- Coroutine
- Custom process
- Common components
- Other features
- Important notices
- Users and cases
- Alternatives
- Sponsor
- License
Features
Built-in Http/WebSocket server
Memory resident
Gracefully reload
Automatically reload after modifying code
Support Laravel/Lumen both, good compatibility
Simple & Out of the box
Benchmark
Which is the fastest web framework?
TechEmpower Framework Benchmarks
Requirements
| Dependency | Requirement |
|---|---|
| PHP | >= 5.5.9 Recommend PHP7+ |
| Swoole | >= 1.7.19 No longer support PHP5 since 2.0.12 Recommend 4.5.0+ |
| Laravel/Lumen | >= 5.1 Recommend 8.0+ |
Install
1.Require package via Composer(packagist).
composer require "hhxsv5/laravel-s:~3.7.0" -vvv
# Make sure that your composer.lock file is under the VCS2.Register service provider(pick one of two).
Laravel: in config/app.php file, Laravel 5.5+ supports package discovery automatically, you should skip this step
'providers' => [
//...
Hhxsv5\LaravelS\Illuminate\LaravelSServiceProvider::class,
],Lumen: in bootstrap/app.php file
$app->register(Hhxsv5\LaravelS\Illuminate\LaravelSServiceProvider::class);3.Publish configuration and binaries.
After upgrading LaravelS, you need to republish; click here to see the change notes of each version.
php artisan laravels publish
# Configuration: config/laravels.php
# Binary: bin/laravels bin/fswatch bin/inotify4.Change config/laravels.php: listen_ip, listen_port, refer Settings.
5.Performance tuning
Number of Workers: LaravelS uses Swoole's Synchronous IO mode, the larger the worker_num setting, the better the concurrency performance, but it will cause more memory usage and process switching overhead. If one request takes 100ms, in order to provide 1000QPS concurrency, at least 100 Worker processes need to be configured. The calculation method is: worker_num = 1000QPS/(1s/1ms) = 100, so incremental pressure testing is needed to calculate the best worker_num.
Run
Please read the notices carefully before running, Important notices(IMPORTANT).
- Commands:
php bin/laravels {start|stop|restart|reload|info|help}.
| Command | Description |
|---|---|
| start | Start LaravelS, list the processes by "ps -ef|grep laravels" |
| stop | Stop LaravelS, and trigger the method onStop of Custom process |
| restart | Restart LaravelS: Stop gracefully before starting; The service is unavailable until startup is complete |
| reload | Reload all Task/Worker/Timer processes which contain your business codes, and trigger the method onReload of Custom process, CANNOT reload Master/Manger processes. After modifying config/laravels.php, you only have to call restart to restart |
| info | Display component version information |
| help | Display help information |
- Boot options for the commands
startandrestart.
| Option | Description |
|---|---|
| -d|--daemonize | Run as a daemon, this option will override the swoole.daemonize setting in laravels.php |
| -e|--env | The environment the command should run under, such as --env=testing will use the configuration file .env.testing firstly, this feature requires Laravel 5.2+ |
| -i|--ignore | Ignore checking PID file of Master process |
| -x|--x-version | The version(branch) of the current project, stored in $_ENV/$_SERVER, access via $_ENV['X_VERSION'] $_SERVER['X_VERSION'] $request->server->get('X_VERSION') |
Runtimefiles:startwill automatically executephp artisan laravels configand generate these files, developers generally don't need to pay attention to them, it's recommended to add them to.gitignore.
| File | Description |
|---|---|
| storage/laravels.conf | LaravelS's runtime configuration file |
| storage/laravels.pid | PID file of Master process |
| storage/laravels-timer-process.pid | PID file of the Timer process |
| storage/laravels-custom-processes.pid | PID file of all custom processes |
Deploy
It is recommended to supervise the main process through Supervisord, the premise is without option
-dand to setswoole.daemonizetofalse.
[program:laravel-s-test]
directory=/var/www/laravel-s-test
command=/usr/local/bin/php bin/laravels start -i
numprocs=1
autostart=true
autorestart=true
startretries=3
user=www-data
redirect_stderr=true
stdout_logfile=/var/log/supervisor/%(program_name)s.log
Cooperate with Nginx (Recommended)
Demo.
gzip on;
gzip_min_length 1024;
gzip_comp_level 2;
gzip_types text/plain text/css text/javascript application/json application/javascript application/x-javascript application/xml application/x-httpd-php image/jpeg image/gif image/png font/ttf font/otf image/svg+xml;
gzip_vary on;
gzip_disable "msie6";
upstream swoole {
# Connect IP:Port
server 127.0.0.1:5200 weight=5 max_fails=3 fail_timeout=30s;
# Connect UnixSocket Stream file, tips: put the socket file in the /dev/shm directory to get better performance
#server unix:/yourpath/laravel-s-test/storage/laravels.sock weight=5 max_fails=3 fail_timeout=30s;
#server 192.168.1.1:5200 weight=3 max_fails=3 fail_timeout=30s;
#server 192.168.1.2:5200 backup;
keepalive 16;
}
server {
listen 80;
# Don't forget to bind the host
server_name laravels.com;
root /yourpath/laravel-s-test/public;
access_log /yourpath/log/nginx/$server_name.access.log main;
autoindex off;
index index.html index.htm;
# Nginx handles the static resources(recommend enabling gzip), LaravelS handles the dynamic resource.
location / {
try_files $uri @laravels;
}
# Response 404 directly when request the PHP file, to avoid exposing public/*.php
#location ~* \.php$ {
# return 404;
#}
location @laravels {
# proxy_connect_timeout 60s;
# proxy_send_timeout 60s;
# proxy_read_timeout 120s;
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Real-PORT $remote_port;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_set_header Scheme $scheme;
proxy_set_header Server-Protocol $server_protocol;
proxy_set_header Server-Name $server_name;
proxy_set_header Server-Addr $server_addr;
proxy_set_header Server-Port $server_port;
# "swoole" is the upstream
proxy_pass http://swoole;
}
}Cooperate with Apache
LoadModule proxy_module /yourpath/modules/mod_proxy.so
LoadModule proxy_balancer_module /yourpath/modules/mod_proxy_balancer.so
LoadModule lbmethod_byrequests_module /yourpath/modules/mod_lbmethod_byrequests.so
LoadModule proxy_http_module /yourpath/modules/mod_proxy_http.so
LoadModule slotmem_shm_module /yourpath/modules/mod_slotmem_shm.so
LoadModule rewrite_module /yourpath/modules/mod_rewrite.so
LoadModule remoteip_module /yourpath/modules/mod_remoteip.so
LoadModule deflate_module /yourpath/modules/mod_deflate.so
<IfModule deflate_module>
SetOutputFilter DEFLATE
DeflateCompressionLevel 2
AddOutputFilterByType DEFLATE text/html text/plain text/css text/javascript application/json application/javascript application/x-javascript application/xml application/x-httpd-php image/jpeg image/gif image/png font/ttf font/otf image/svg+xml
</IfModule>
<VirtualHost *:80>
# Don't forget to bind the host
ServerName www.laravels.com
ServerAdmin hhxsv5@sina.com
DocumentRoot /yourpath/laravel-s-test/public;
DirectoryIndex index.html index.htm
<Directory "/">
AllowOverride None
Require all granted
</Directory>
RemoteIPHeader X-Forwarded-For
ProxyRequests Off
ProxyPreserveHost On
<Proxy balancer://laravels>
BalancerMember http://192.168.1.1:5200 loadfactor=7
#BalancerMember http://192.168.1.2:5200 loadfactor=3
#BalancerMember http://192.168.1.3:5200 loadfactor=1 status=+H
ProxySet lbmethod=byrequests
</Proxy>
#ProxyPass / balancer://laravels/
#ProxyPassReverse / balancer://laravels/
# Apache handles the static resources, LaravelS handles the dynamic resource.
RewriteEngine On
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-d
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-f
RewriteRule ^/(.*)$ balancer://laravels%{REQUEST_URI} [P,L]
ErrorLog ${APACHE_LOG_DIR}/www.laravels.com.error.log
CustomLog ${APACHE_LOG_DIR}/www.laravels.com.access.log combined
</VirtualHost>Enable WebSocket server
The Listening address of WebSocket Sever is the same as Http Server.
1.Create WebSocket Handler class, and implement interface WebSocketHandlerInterface.The instant is automatically instantiated when start, you do not need to manually create it.
namespace App\Services;
use Hhxsv5\LaravelS\Swoole\WebSocketHandlerInterface;
use Swoole\Http\Request;
use Swoole\Http\Response;
use Swoole\WebSocket\Frame;
use Swoole\WebSocket\Server;
/**
* @see https://www.swoole.co.uk/docs/modules/swoole-websocket-server
*/
class WebSocketService implements WebSocketHandlerInterface
{
// Declare constructor without parameters
public function __construct()
{
}
// public function onHandShake(Request $request, Response $response)
// {
// Custom handshake: https://www.swoole.co.uk/docs/modules/swoole-websocket-server-on-handshake
// The onOpen event will be triggered automatically after a successful handshake
// }
public function onOpen(Server $server, Request $request)
{
// Before the onOpen event is triggered, the HTTP request to establish the WebSocket has passed the Laravel route,
// so Laravel's Request, Auth information are readable, Session is readable and writable, but only in the onOpen event.
// \Log::info('New WebSocket connection', [$request->fd, request()->all(), session()->getId(), session('xxx'), session(['yyy' => time()])]);
// The exceptions thrown here will be caught by the upper layer and recorded in the Swoole log. Developers need to try/catch manually.
$server->push($request->fd, 'Welcome to LaravelS');
}
public function onMessage(Server $server, Frame $frame)
{
// \Log::info('Received message', [$frame->fd, $frame->data, $frame->opcode, $frame->finish]);
// The exceptions thrown here will be caught by the upper layer and recorded in the Swoole log. Developers need to try/catch manually.
$server->push($frame->fd, date('Y-m-d H:i:s'));
}
public function onClose(Server $server, $fd, $reactorId)
{
// The exceptions thrown here will be caught by the upper layer and recorded in the Swoole log. Developers need to try/catch manually.
}
}2.Modify config/laravels.php.
// ...
'websocket' => [
'enable' => true, // Note: set enable to true
'handler' => \App\Services\WebSocketService::class,
],
'swoole' => [
//...
// Must set dispatch_mode in (2, 4, 5), see https://www.swoole.co.uk/docs/modules/swoole-server/configuration
'dispatch_mode' => 2,
//...
],
// ...3.Use SwooleTable to bind FD & UserId, optional, Swoole Table Demo. Also you can use the other global storage services, like Redis/Memcached/MySQL, but be careful that FD will be possible conflicting between multiple Swoole Servers.
4.Cooperate with Nginx (Recommended)
Refer WebSocket Proxy
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
upstream swoole {
# Connect IP:Port
server 127.0.0.1:5200 weight=5 max_fails=3 fail_timeout=30s;
# Connect UnixSocket Stream file, tips: put the socket file in the /dev/shm directory to get better performance
#server unix:/yourpath/laravel-s-test/storage/laravels.sock weight=5 max_fails=3 fail_timeout=30s;
#server 192.168.1.1:5200 weight=3 max_fails=3 fail_timeout=30s;
#server 192.168.1.2:5200 backup;
keepalive 16;
}
server {
listen 80;
# Don't forget to bind the host
server_name laravels.com;
root /yourpath/laravel-s-test/public;
access_log /yourpath/log/nginx/$server_name.access.log main;
autoindex off;
index index.html index.htm;
# Nginx handles the static resources(recommend enabling gzip), LaravelS handles the dynamic resource.
location / {
try_files $uri @laravels;
}
# Response 404 directly when request the PHP file, to avoid exposing public/*.php
#location ~* \.php$ {
# return 404;
#}
# Http and WebSocket are concomitant, Nginx identifies them by "location"
# !!! The location of WebSocket is "/ws"
# Javascript: var ws = new WebSocket("ws://laravels.com/ws");
location =/ws {
# proxy_connect_timeout 60s;
# proxy_send_timeout 60s;
# proxy_read_timeout: Nginx will close the connection if the proxied server does not send data to Nginx in 60 seconds; At the same time, this close behavior is also affected by heartbeat setting of Swoole.
# proxy_read_timeout 60s;
proxy_http_version 1.1;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Real-PORT $remote_port;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_set_header Scheme $scheme;
proxy_set_header Server-Protocol $server_protocol;
proxy_set_header Server-Name $server_name;
proxy_set_header Server-Addr $server_addr;
proxy_set_header Server-Port $server_port;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_pass http://swoole;
}
location @laravels {
# proxy_connect_timeout 60s;
# proxy_send_timeout 60s;
# proxy_read_timeout 60s;
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Real-PORT $remote_port;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_set_header Scheme $scheme;
proxy_set_header Server-Protocol $server_protocol;
proxy_set_header Server-Name $server_name;
proxy_set_header Server-Addr $server_addr;
proxy_set_header Server-Port $server_port;
proxy_pass http://swoole;
}
}5.Heartbeat setting
Heartbeat setting of Swoole
// config/laravels.php
'swoole' => [
//...
// All connections are traversed every 60 seconds. If a connection does not send any data to the server within 600 seconds, the connection will be forced to close.
'heartbeat_idle_time' => 600,
'heartbeat_check_interval' => 60,
//...
],Proxy read timeout of Nginx
# Nginx will close the connection if the proxied server does not send data to Nginx in 60 seconds
proxy_read_timeout 60s;6.Push data in controller
namespace App\Http\Controllers;
class TestController extends Controller
{
public function push()
{
$fd = 1; // Find fd by userId from a map [userId=>fd].
/**@var \Swoole\WebSocket\Server $swoole */
$swoole = app('swoole');
$success = $swoole->push($fd, 'Push data to fd#1 in Controller');
var_dump($success);
}
}Listen events
System events
Usually, you can reset/destroy some
global/staticvariables, or change the currentRequest/Responseobject.
laravels.received_request After LaravelS parsed Swoole\Http\Request to Illuminate\Http\Request, before Laravel's Kernel handles this request.
// Edit file `app/Providers/EventServiceProvider.php`, add the following code into method `boot`
// If no variable $events, you can also call Facade \Event::listen().
$events->listen('laravels.received_request', function (\Illuminate\Http\Request $req, $app) {
$req->query->set('get_key', 'hhxsv5');// Change query of request
$req->request->set('post_key', 'hhxsv5'); // Change post of request
});laravels.generated_response After Laravel's Kernel handled the request, before LaravelS parses Illuminate\Http\Response to Swoole\Http\Response.
// Edit file `app/Providers/EventServiceProvider.php`, add the following code into method `boot`
// If no variable $events, you can also call Facade \Event::listen().
$events->listen('laravels.generated_response', function (\Illuminate\Http\Request $req, \Symfony\Component\HttpFoundation\Response $rsp, $app) {
$rsp->headers->set('header-key', 'hhxsv5');// Change header of response
});Customized asynchronous events
This feature depends on
AsyncTaskofSwoole, your need to setswoole.task_worker_numinconfig/laravels.phpfirstly. The performance of asynchronous event processing is influenced by number of Swoole task process, you need to set task_worker_num appropriately.
1.Create event class.
use Hhxsv5\LaravelS\Swoole\Task\Event;
class TestEvent extends Event
{
protected $listeners = [
// Listener list
TestListener1::class,
// TestListener2::class,
];
private $data;
public function __construct($data)
{
$this->data = $data;
}
public function getData()
{
return $this->data;
}
}2.Create listener class.
use Hhxsv5\LaravelS\Swoole\Task\Task;
use Hhxsv5\LaravelS\Swoole\Task\Listener;
class TestListener1 extends Listener
{
/**
* @var TestEvent
*/
protected $event;
public function handle()
{
\Log::info(__CLASS__ . ':handle start', [$this->event->getData()]);
sleep(2);// Simulate the slow codes
// Deliver task in CronJob, but NOT support callback finish() of task.
// Note: Modify task_ipc_mode to 1 or 2 in config/laravels.php, see https://www.swoole.co.uk/docs/modules/swoole-server/configuration
$ret = Task::deliver(new TestTask('task data'));
var_dump($ret);
// The exceptions thrown here will be caught by the upper layer and recorded in the Swoole log. Developers need to try/catch manually.
}
}3.Fire event.
// Create instance of event and fire it, "fire" is asynchronous.
use Hhxsv5\LaravelS\Swoole\Task\Event;
$event = new TestEvent('event data');
// $event->delay(10); // Delay 10 seconds to fire event
// $event->setTries(3); // When an error occurs, try 3 times in total
$success = Event::fire($event);
var_dump($success);// Return true if sucess, otherwise falseAsynchronous task queue
This feature depends on
AsyncTaskofSwoole, your need to setswoole.task_worker_numinconfig/laravels.phpfirstly. The performance of task processing is influenced by number of Swoole task process, you need to set task_worker_num appropriately.
1.Create task class.
use Hhxsv5\LaravelS\Swoole\Task\Task;
class TestTask extends Task
{
private $data;
private $result;
public function __construct($data)
{
$this->data = $data;
}
// The logic of task handling, run in task process, CAN NOT deliver task
public function handle()
{
\Log::info(__CLASS__ . ':handle start', [$this->data]);
sleep(2);// Simulate the slow codes
// The exceptions thrown here will be caught by the upper layer and recorded in the Swoole log. Developers need to try/catch manually.
$this->result = 'the result of ' . $this->data;
}
// Optional, finish event, the logic of after task handling, run in worker process, CAN deliver task
public function finish()
{
\Log::info(__CLASS__ . ':finish start', [$this->result]);
Task::deliver(new TestTask2('task2 data')); // Deliver the other task
}
}2.Deliver task.
// Create instance of TestTask and deliver it, "deliver" is asynchronous.
use Hhxsv5\LaravelS\Swoole\Task\Task;
$task = new TestTask('task data');
// $task->delay(3);// delay 3 seconds to deliver task
// $task->setTries(3); // When an error occurs, try 3 times in total
$ret = Task::deliver($task);
var_dump($ret);// Return true if sucess, otherwise falseMillisecond cron job
Wrapper cron job base on Swoole's Millisecond Timer, replace
LinuxCrontab.
1.Create cron job class.
namespace App\Jobs\Timer;
use App\Tasks\TestTask;
use Swoole\Coroutine;
use Hhxsv5\LaravelS\Swoole\Task\Task;
use Hhxsv5\LaravelS\Swoole\Timer\CronJob;
class TestCronJob extends CronJob
{
protected $i = 0;
// !!! The `interval` and `isImmediate` of cron job can be configured in two ways(pick one of two): one is to overload the corresponding method, and the other is to pass parameters when registering cron job.
// --- Override the corresponding method to return the configuration: begin
public function interval()
{
return 1000;// Run every 1000ms
}
public function isImmediate()
{
return false;// Whether to trigger `run` immediately after setting up
}
// --- Override the corresponding method to return the configuration: end
public function run()
{
\Log::info(__METHOD__, ['start', $this->i, microtime(true)]);
// do something
// sleep(1); // Swoole < 2.1
Coroutine::sleep(1); // Swoole>=2.1 Coroutine will be automatically created for run().
$this->i++;
\Log::info(__METHOD__, ['end', $this->i, microtime(true)]);
if ($this->i >= 10) { // Run 10 times only
\Log::info(__METHOD__, ['stop', $this->i, microtime(true)]);
$this->stop(); // Stop this cron job, but it will run again after restart/reload.
// Deliver task in CronJob, but NOT support callback finish() of task.
// Note: Modify task_ipc_mode to 1 or 2 in config/laravels.php, see https://www.swoole.co.uk/docs/modules/swoole-server/configuration
$ret = Task::deliver(new TestTask('task data'));
var_dump($ret);
}
// The exceptions thrown here will be caught by the upper layer and recorded in the Swoole log. Developers need to try/catch manually.
}
}2.Register cron job.
// Register cron jobs in file "config/laravels.php"
[
// ...
'timer' => [
'enable' => true, // Enable Timer
'jobs' => [ // The list of cron job
// Enable LaravelScheduleJob to run `php artisan schedule:run` every 1 minute, replace Linux Crontab
// \Hhxsv5\LaravelS\Illuminate\LaravelScheduleJob::class,
// Two ways to configure parameters:
// [\App\Jobs\Timer\TestCronJob::class, [1000, true]], // Pass in parameters when registering
\App\Jobs\Timer\TestCronJob::class, // Override the corresponding method to return the configuration
],
'max_wait_time' => 5, // Max waiting time of reloading
// Enable the global lock to ensure that only one instance starts the timer when deploying multiple instances. This feature depends on Redis, please see https://laravel.com/docs/7.x/redis
'global_lock' => false,
'global_lock_key' => config('app.name', 'Laravel'),
],
// ...
];3.Note: it will launch multiple timers when build the server cluster, so you need to make sure that launch one timer only to avoid running repetitive task.
4.LaravelS v3.4.0 starts to support the hot restart [Reload] Timer process. After LaravelS receives the SIGUSR1 signal, it waits for max_wait_time(default 5) seconds to end the process, then the Manager process will pull up the Timer process again.
5.If you only need to use minute-level scheduled tasks, it is recommended to enable Hhxsv5\LaravelS\Illuminate\LaravelScheduleJob instead of Linux Crontab, so that you can follow the coding habits of Laravel task scheduling and configure Kernel.
// app/Console/Kernel.php
protected function schedule(Schedule $schedule)
{
// runInBackground() will start a new child process to execute the task. This is asynchronous and will not affect the execution timing of other tasks.
$schedule->command(TestCommand::class)->runInBackground()->everyMinute();
}Automatically reload after modifying code
Via inotify, support Linux only.
1.Install inotify extension.
2.Turn on the switch in Settings.
3.Notice: Modify the file only in Linux to receive the file change events. It's recommended to use the latest Docker. Vagrant Solution.
Via fswatch, support OS X/Linux/Windows.
1.Install fswatch.
2.Run command in your project root directory.
# Watch current directory
./bin/fswatch
# Watch app directory
./bin/fswatch ./appVia inotifywait, support Linux.
1.Install inotify-tools.
2.Run command in your project root directory.
# Watch current directory
./bin/inotify
# Watch app directory
./bin/inotify ./appWhen the above methods does not work, the ultimate solution: set max_request=1,worker_num=1, so that Worker process will restart after processing a request. The performance of this method is very poor, so only development environment use.
Get the instance of SwooleServer in your project
/**
* $swoole is the instance of `Swoole\WebSocket\Server` if enable WebSocket server, otherwise `Swoole\Http\Server`
* @var \Swoole\WebSocket\Server|\Swoole\Http\Server $swoole
*/
$swoole = app('swoole');
var_dump($swoole->stats());
$swoole->push($fd, 'Push WebSocket message');Use SwooleTable
1.Define Table, support multiple.
All defined tables will be created before Swoole starting.
// in file "config/laravels.php"
[
// ...
'swoole_tables' => [
// Scene：bind UserId & FD in WebSocket
'ws' => [// The Key is table name, will add suffix "Table" to avoid naming conflicts. Here defined a table named "wsTable"
'size' => 102400,// The max size
'column' => [// Define the columns
['name' => 'value', 'type' => \Swoole\Table::TYPE_INT, 'size' => 8],
],
],
//...Define the other tables
],
// ...
];2.Access Table: all table instances will be bound on SwooleServer, access by app('swoole')->xxxTable.
namespace App\Services;
use Hhxsv5\LaravelS\Swoole\WebSocketHandlerInterface;
use Swoole\Http\Request;
use Swoole\WebSocket\Frame;
use Swoole\WebSocket\Server;
class WebSocketService implements WebSocketHandlerInterface
{
/**@var \Swoole\Table $wsTable */
private $wsTable;
public function __construct()
{
$this->wsTable = app('swoole')->wsTable;
}
// Scene：bind UserId & FD in WebSocket
public function onOpen(Server $server, Request $request)
{
// var_dump(app('swoole') === $server);// The same instance
/**
* Get the currently logged in user
* This feature requires that the path to establish a WebSocket connection go through middleware such as Authenticate.
* E.g:
* Browser side: var ws = new WebSocket("ws://127.0.0.1:5200/ws");
* Then the /ws route in Laravel needs to add the middleware like Authenticate.
* Route::get('/ws', function () {
* // Respond any content with status code 200
* return 'websocket';
* })->middleware(['auth']);
*/
// $user = Auth::user();
// $userId = $user ? $user->id : 0; // 0 means a guest user who is not logged in
$userId = mt_rand(1000, 10000);
// if (!$userId) {
// // Disconnect the connections of unlogged users
// $server->disconnect($request->fd);
// return;
// }
$this->wsTable->set('uid:' . $userId, ['value' => $request->fd]);// Bind map uid to fd
$this->wsTable->set('fd:' . $request->fd, ['value' => $userId]);// Bind map fd to uid
$server->push($request->fd, "Welcome to LaravelS #{$request->fd}");
}
public function onMessage(Server $server, Frame $frame)
{
// Broadcast
foreach ($this->wsTable as $key => $row) {
if (strpos($key, 'uid:') === 0 && $server->isEstablished($row['value'])) {
$content = sprintf('Broadcast: new message "%s" from #%d', $frame->data, $frame->fd);
$server->push($row['value'], $content);
}
}
}
public function onClose(Server $server, $fd, $reactorId)
{
$uid = $this->wsTable->get('fd:' . $fd);
if ($uid !== false) {
$this->wsTable->del('uid:' . $uid['value']); // Unbind uid map
}
$this->wsTable->del('fd:' . $fd);// Unbind fd map
$server->push($fd, "Goodbye #{$fd}");
}
}Multi-port mixed protocol
For more information, please refer to Swoole Server AddListener
To make our main server support more protocols not just Http and WebSocket, we bring the feature multi-port mixed protocol of Swoole in LaravelS and name it Socket. Now, you can build TCP/UDP applications easily on top of Laravel.
Create Socket handler class, and extend Hhxsv5\LaravelS\Swoole\Socket\{TcpSocket|UdpSocket|Http|WebSocket}.
namespace App\Sockets;
use Hhxsv5\LaravelS\Swoole\Socket\TcpSocket;
use Swoole\Server;
class TestTcpSocket extends TcpSocket
{
public function onConnect(Server $server, $fd, $reactorId)
{
\Log::info('New TCP connection', [$fd]);
$server->send($fd, 'Welcome to LaravelS.');
}
public function onReceive(Server $server, $fd, $reactorId, $data)
{
\Log::info('Received data', [$fd, $data]);
$server->send($fd, 'LaravelS: ' . $data);
if ($data === "quit\r\n") {
$server->send($fd, 'LaravelS: bye' . PHP_EOL);
$server->close($fd);
}
}
public function onClose(Server $server, $fd, $reactorId)
{
\Log::info('Close TCP connection', [$fd]);
$server->send($fd, 'Goodbye');
}
}These Socket connections share the same worker processes with your HTTP/WebSocket connections. So it won't be a problem at all if you want to deliver tasks, use SwooleTable, even Laravel components such as DB, Eloquent and so on. At the same time, you can access Swoole\Server\Port object directly by member property swoolePort.
public function onReceive(Server $server, $fd, $reactorId, $data)
{
$port = $this->swoolePort; // Get the `Swoole\Server\Port` object
}namespace App\Http\Controllers;
class TestController extends Controller
{
public function test()
{
/**@var \Swoole\Http\Server|\Swoole\WebSocket\Server $swoole */
$swoole = app('swoole');
// $swoole->ports: Traverse all Port objects, https://www.swoole.co.uk/docs/modules/swoole-server/multiple-ports
$port = $swoole->ports[0]; // Get the `Swoole\Server\Port` object, $port[0] is the port of the main server
foreach ($port->connections as $fd) { // Traverse all connections
// $swoole->send($fd, 'Send tcp message');
// if($swoole->isEstablished($fd)) {
// $swoole->push($fd, 'Send websocket message');
// }
}
}
}Register Sockets.
// Edit `config/laravels.php`
//...
'sockets' => [
[
'host' => '127.0.0.1',
'port' => 5291,
'type' => SWOOLE_SOCK_TCP,// Socket type: SWOOLE_SOCK_TCP/SWOOLE_SOCK_TCP6/SWOOLE_SOCK_UDP/SWOOLE_SOCK_UDP6/SWOOLE_UNIX_DGRAM/SWOOLE_UNIX_STREAM
'settings' => [// Swoole settings：https://www.swoole.co.uk/docs/modules/swoole-server-methods#swoole_server-addlistener
'open_eof_check' => true,
'package_eof' => "\r\n",
],
'handler' => \App\Sockets\TestTcpSocket::class,
'enable' => true, // whether to enable, default true
],
],About the heartbeat configuration, it can only be set on the main server and cannot be configured on Socket, but the Socket inherits the heartbeat configuration of the main server.
For TCP socket, onConnect and onClose events will be blocked when dispatch_mode of Swoole is 1/3, so if you want to unblock these two events please set dispatch_mode to 2/4/5.
'swoole' => [
//...
'dispatch_mode' => 2,
//...
];Test.
TCP: telnet 127.0.0.1 5291
UDP: [Linux] echo "Hello LaravelS" > /dev/udp/127.0.0.1/5292
Register example of other protocols.
- UDP
'sockets' => [
[
'host' => '0.0.0.0',
'port' => 5292,
'type' => SWOOLE_SOCK_UDP,
'settings' => [
'open_eof_check' => true,
'package_eof' => "\r\n",
],
'handler' => \App\Sockets\TestUdpSocket::class,
],
],- Http
'sockets' => [
[
'host' => '0.0.0.0',
'port' => 5293,
'type' => SWOOLE_SOCK_TCP,
'settings' => [
'open_http_protocol' => true,
],
'handler' => \App\Sockets\TestHttp::class,
],
],- WebSocket: The main server must
turn on WebSocket, that is, setwebsocket.enabletotrue.
'sockets' => [
[
'host' => '0.0.0.0',
'port' => 5294,
'type' => SWOOLE_SOCK_TCP,
'settings' => [
'open_http_protocol' => true,
'open_websocket_protocol' => true,
],
'handler' => \App\Sockets\TestWebSocket::class,
],
],Coroutine
Warning: The order of code execution in the coroutine is out of order. The data of the request level should be isolated by the coroutine ID. However, there are many singleton and static attributes in Laravel/Lumen, the data between different requests will affect each other, it's Unsafe. For example, the database connection is a singleton, the same database connection shares the same PDO resource. This is fine in the synchronous blocking mode, but it does not work in the asynchronous coroutine mode. Each query needs to create different connections and maintain IO state of different connections, which requires a connection pool.
DO NOT enable the coroutine, only the custom process can use the coroutine.
Custom process
Support developers to create special work processes for monitoring, reporting, or other special tasks. Refer addProcess.
Create Proccess class, implements CustomProcessInterface.
namespace App\Processes;
use App\Tasks\TestTask;
use Hhxsv5\LaravelS\Swoole\Process\CustomProcessInterface;
use Hhxsv5\LaravelS\Swoole\Task\Task;
use Swoole\Coroutine;
use Swoole\Http\Server;
use Swoole\Process;
class TestProcess implements CustomProcessInterface
{
/**
* @var bool Quit tag for Reload updates
*/
private static $quit = false;
public static function callback(Server $swoole, Process $process)
{
// The callback method cannot exit. Once exited, Manager process will automatically create the process
while (!self::$quit) {
\Log::info('Test process: running');
// sleep(1); // Swoole < 2.1
Coroutine::sleep(1); // Swoole>=2.1: Coroutine & Runtime will be automatically enabled for callback().
// Deliver task in custom process, but NOT support callback finish() of task.
// Note: Modify task_ipc_mode to 1 or 2 in config/laravels.php, see https://www.swoole.co.uk/docs/modules/swoole-server/configuration
$ret = Task::deliver(new TestTask('task data'));
var_dump($ret);
// The upper layer will catch the exception thrown in the callback and record it in the Swoole log, and then this process will exit. The Manager process will re-create the process after 3 seconds, so developers need to try/catch to catch the exception by themselves to avoid frequent process creation.
// throw new \Exception('an exception');
}
}
// Requirements: LaravelS >= v3.4.0 & callback() must be async non-blocking program.
public static function onReload(Server $swoole, Process $process)
{
// Stop the process...
// Then end process
\Log::info('Test process: reloading');
self::$quit = true;
// $process->exit(0); // Force exit process
}
// Requirements: LaravelS >= v3.7.4 & callback() must be async non-blocking program.
public static function onStop(Server $swoole, Process $process)
{
// Stop the process...
// Then end process
\Log::info('Test process: stopping');
self::$quit = true;
// $process->exit(0); // Force exit process
}
}Register TestProcess.
// Edit `config/laravels.php`
// ...
'processes' => [
'test' => [ // Key name is process name
'class' => \App\Processes\TestProcess::class,
'redirect' => false, // Whether redirect stdin/stdout, true or false
'pipe' => 0, // The type of pipeline, 0: no pipeline 1: SOCK_STREAM 2: SOCK_DGRAM
'enable' => true, // Whether to enable, default true
//'num' => 3 // To create multiple processes of this class, default is 1
//'queue' => [ // Enable message queue as inter-process communication, configure empty array means use default parameters
// 'msg_key' => 0, // The key of the message queue. Default: ftok(__FILE__, 1).
// 'mode' => 2, // Communication mode, default is 2, which means contention mode
// 'capacity' => 8192, // The length of a single message, is limited by the operating system kernel parameters. The default is 8192, and the maximum is 65536
//],
//'restart_interval' => 5, // After the process exits abnormally, how many seconds to wait before restarting the process, default 5 seconds
],
],Note: The callback() cannot quit. If quit, the Manager process will re-create the process.
Example: Write data to a custom process.
// config/laravels.php
'processes' => [
'test' => [
'class' => \App\Processes\TestProcess::class,
'redirect' => false,
'pipe' => 1,
],
],// app/Processes/TestProcess.php
public static function callback(Server $swoole, Process $process)
{
while ($data = $process->read()) {
\Log::info('TestProcess: read data', [$data]);
$process->write('TestProcess: ' . $data);
}
}// app/Http/Controllers/TestController.php
public function testProcessWrite()
{
/**@var \Swoole\Process $process */
$process = app('swoole')->customProcesses['test'];
$process->write('TestController: write data' . time());
var_dump($process->read());
}Common components
Apollo
LaravelSwill pull theApolloconfiguration and write it to the.envfile when starting. At the same time,LaravelSwill start the custom processapolloto monitor the configuration and automaticallyreloadwhen the configuration changes.
Enable Apollo: add --enable-apollo and Apollo parameters to the startup parameters.
php bin/laravels start --enable-apollo --apollo-server=http://127.0.0.1:8080 --apollo-app-id=LARAVEL-S-TESTSupport hot updates(optional).
// Edit `config/laravels.php`
'processes' => Hhxsv5\LaravelS\Components\Apollo\Process::getDefinition(),// When there are other custom process configurations
'processes' => [
'test' => [
'class' => \App\Processes\TestProcess::class,
'redirect' => false,
'pipe' => 1,
],
// ...
] + Hhxsv5\LaravelS\Components\Apollo\Process::getDefinition(),List of available parameters.
| Parameter | Description | Default | Demo |
|---|---|---|---|
| apollo-server | Apollo server URL | - | --apollo-server=http://127.0.0.1:8080 |
| apollo-app-id | Apollo APP ID | - | --apollo-app-id=LARAVEL-S-TEST |
| apollo-namespaces | The namespace to which the APP belongs, support specify the multiple | application | --apollo-namespaces=application --apollo-namespaces=env |
| apollo-cluster | The cluster to which the APP belongs | default | --apollo-cluster=default |
| apollo-client-ip | IP of current instance, can also be used for grayscale publishing | Local intranet IP | --apollo-client-ip=10.2.1.83 |
| apollo-pull-timeout | Timeout time(seconds) when pulling configuration | 5 | --apollo-pull-timeout=5 |
| apollo-backup-old-env | Whether to backup the old configuration file when updating the configuration file .env | false | --apollo-backup-old-env |
Prometheus
Support Prometheus monitoring and alarm, Grafana visually view monitoring metrics. Please refer to Docker Compose for the environment construction of Prometheus and Grafana.
Require extension APCu >= 5.0.0, please install it by pecl install apcu.
Copy the configuration file prometheus.php to the config directory of your project. Modify the configuration as appropriate.
# Execute commands in the project root directory
cp vendor/hhxsv5/laravel-s/config/prometheus.php config/If your project is Lumen, you also need to manually load the configuration $app->configure('prometheus'); in bootstrap/app.php.
Configure global middleware: Hhxsv5\LaravelS\Components\Prometheus\RequestMiddleware::class. In order to count the request time consumption as accurately as possible, RequestMiddleware must be the first global middleware, which needs to be placed in front of other middleware.
Register ServiceProvider: Hhxsv5\LaravelS\Components\Prometheus\ServiceProvider::class.
Configure the CollectorProcess in config/laravels.php to collect the metrics of Swoole Worker/Task/Timer processes regularly.
'processes' => Hhxsv5\LaravelS\Components\Prometheus\CollectorProcess::getDefinition(),Create the route to output metrics.
use Hhxsv5\LaravelS\Components\Prometheus\Exporter;
Route::get('/actuator/prometheus', function () {
$result = app(Exporter::class)->render();
return response($result, 200, ['Content-Type' => Exporter::REDNER_MIME_TYPE]);
});Complete the configuration of Prometheus and start it.
global:
scrape_interval: 5s
scrape_timeout: 5s
evaluation_interval: 30s
scrape_configs:
- job_name: laravel-s-test
honor_timestamps: true
metrics_path: /actuator/prometheus
scheme: http
follow_redirects: true
static_configs:
- targets:
- 127.0.0.1:5200 # The ip and port of the monitored service
# Dynamically discovered using one of the supported service-discovery mechanisms
# https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config
# - job_name: laravels-eureka
# honor_timestamps: true
# scrape_interval: 5s
# metrics_path: /actuator/prometheus
# scheme: http
# follow_redirects: true
# eureka_sd_configs:
# - server: http://127.0.0.1:8080/eureka
# follow_redirects: true
# refresh_interval: 5sStart Grafana, then import panel json.
Other features
Configure Swoole events
Supported events:
| Event | Interface | When happened |
|---|---|---|
| ServerStart | Hhxsv5\LaravelS\Swoole\Events\ServerStartInterface | Occurs when the Master process is starting, this event should not handle complex business logic, and can only do some simple work of initialization. |
| ServerStop | Hhxsv5\LaravelS\Swoole\Events\ServerStopInterface | Occurs when the server exits normally, CANNOT use async or coroutine related APIs in this event. |
| WorkerStart | Hhxsv5\LaravelS\Swoole\Events\WorkerStartInterface | Occurs after the Worker/Task process is started, and the Laravel initialization has been completed. |
| WorkerStop | Hhxsv5\LaravelS\Swoole\Events\WorkerStopInterface | Occurs after the Worker/Task process exits normally |
| WorkerError | Hhxsv5\LaravelS\Swoole\Events\WorkerErrorInterface | Occurs when an exception or fatal error occurs in the Worker/Task process |
1.Create an event class to implement the corresponding interface.
namespace App\Events;
use Hhxsv5\LaravelS\Swoole\Events\ServerStartInterface;
use Swoole\Atomic;
use Swoole\Http\Server;
class ServerStartEvent implements ServerStartInterface
{
public function __construct()
{
}
public function handle(Server $server)
{
// Initialize a global counter (available across processes)
$server->atomicCount = new Atomic(2233);
// Invoked in controller: app('swoole')->atomicCount->get();
}
}namespace App\Events;
use Hhxsv5\LaravelS\Swoole\Events\WorkerStartInterface;
use Swoole\Http\Server;
class WorkerStartEvent implements WorkerStartInterface
{
public function __construct()
{
}
public function handle(Server $server, $workerId)
{
// Initialize a database connection pool
// DatabaseConnectionPool::init();
}
}2.Configuration.
// Edit `config/laravels.php`
'event_handlers' => [
'ServerStart' => [\App\Events\ServerStartEvent::class], // Trigger events in array order
'WorkerStart' => [\App\Events\WorkerStartEvent::class],
],Serverless
Alibaba Cloud Function Compute
1.Modify bootstrap/app.php and set the storage directory. Because the project directory is read-only, the /tmp directory can only be read and written.
$app->useStoragePath(env('APP_STORAGE_PATH', '/tmp/storage'));2.Create a shell script laravels_bootstrap and grant executable permission.
#!/usr/bin/env bash
set +e
# Create storage-related directories
mkdir -p /tmp/storage/app/public
mkdir -p /tmp/storage/framework/cache
mkdir -p /tmp/storage/framework/sessions
mkdir -p /tmp/storage/framework/testing
mkdir -p /tmp/storage/framework/views
mkdir -p /tmp/storage/logs
# Set the environment variable APP_STORAGE_PATH, please make sure it's the same as APP_STORAGE_PATH in .env
export APP_STORAGE_PATH=/tmp/storage
# Start LaravelS
php bin/laravels start3.Configure template.xml.
ROSTemplateFormatVersion: '2015-09-01'
Transform: 'Aliyun::Serverless-2018-04-03'
Resources:
laravel-s-demo:
Type: 'Aliyun::Serverless::Service'
Properties:
Description: 'LaravelS Demo for Serverless'
fc-laravel-s:
Type: 'Aliyun::Serverless::Function'
Properties:
Handler: laravels.handler
Runtime: custom
MemorySize: 512
Timeout: 30
CodeUri: ./
InstanceConcurrency: 10
EnvironmentVariables:
BOOTSTRAP_FILE: laravels_bootstrap
Important notices
Singleton Issue
Under FPM mode, singleton instances will be instantiated and recycled in every request, request start=>instantiate instance=>request end=>recycled instance.
Under Swoole Server, All singleton instances will be held in memory, different lifetime from FPM, request start=>instantiate instance=>request end=>do not recycle singleton instance. So need developer to maintain status of singleton instances in every request.
Common solutions:
Write a XxxCleaner class to clean up the singleton object state. This class implements the interface Hhxsv5\LaravelS\Illuminate\Cleaners\CleanerInterface and then registers it in cleaners of laravels.php.
Reset status of singleton instances by Middleware.
Re-register ServiceProvider, add XxxServiceProvider into register_providers of file laravels.php. So that reinitialize singleton instances in every request Refer.
Cleaners
Known issues
Known issues: a package of known issues and solutions.
Debugging method
Logging; if you want to output to the console, you can use stderr, Log::channel('stderr')->debug('debug message').
Laravel Dump Server(Laravel 5.7 has been integrated by default).
Read request
Read request by Illuminate\Http\Request Object, $_ENV is readable, $_SERVER is partially readable, CANNOT USE $_GET/$_POST/$_FILES/$_COOKIE/$_REQUEST/$_SESSION/$GLOBALS.
public function form(\Illuminate\Http\Request $request)
{
$name = $request->input('name');
$all = $request->all();
$sessionId = $request->cookie('sessionId');
$photo = $request->file('photo');
// Call getContent() to get the raw POST body, instead of file_get_contents('php://input')
$rawContent = $request->getContent();
//...
}Output response
Respond by Illuminate\Http\Response Object, compatible with echo/vardump()/print_r()，CANNOT USE functions dd()/exit()/die()/header()/setcookie()/http_response_code().
public function json()
{
return response()->json(['time' => time()])->header('header1', 'value1')->withCookie('c1', 'v1');
}Persistent connection
Singleton connection will be resident in memory, it is recommended to turn on persistent connection for better performance.
- Database connection, it
willreconnect automaticallyimmediatelyafter disconnect.
// config/database.php
'connections' => [
'my_conn' => [
'driver' => 'mysql',
'host' => env('DB_MY_CONN_HOST', 'localhost'),
'port' => env('DB_MY_CONN_PORT', 3306),
'database' => env('DB_MY_CONN_DATABASE', 'forge'),
'username' => env('DB_MY_CONN_USERNAME', 'forge'),
'password' => env('DB_MY_CONN_PASSWORD', ''),
'charset' => 'utf8mb4',
'collation' => 'utf8mb4_unicode_ci',
'prefix' => '',
'strict' => false,
'options' => [
// Enable persistent connection
\PDO::ATTR_PERSISTENT => true,
],
],
],- Redis connection, it
won'treconnect automaticallyimmediatelyafter disconnect, and will throw an exception about lost connection, reconnect next time. You need to make sure thatSELECT DBcorrectly before operating Redis every time.
// config/database.php
'redis' => [
'client' => env('REDIS_CLIENT', 'phpredis'), // It is recommended to use phpredis for better performance.
'default' => [
'host' => env('REDIS_HOST', 'localhost'),
'password' => env('REDIS_PASSWORD', null),
'port' => env('REDIS_PORT', 6379),
'database' => 0,
'persistent' => true, // Enable persistent connection
],
],About memory leaks
Avoid using global variables. If necessary, please clean or reset them manually.
Infinitely appending element into static/global variable will lead to OOM(Out of Memory).
class Test
{
public static $array = [];
public static $string = '';
}
// Controller
public function test(Request $req)
{
// Out of Memory
Test::$array[] = $req->input('param1');
Test::$string .= $req->input('param2');
}Memory leak detection method
Modify config/laravels.php: worker_num=1, max_request=1000000, remember to change it back after test;
Add routing /debug-memory-leak without route middleware to observe the memory changes of the Worker process;
Route::get('/debug-memory-leak', function () {
global $previous;
$current = memory_get_usage();
$stats = [
'prev_mem' => $previous,
'curr_mem' => $current,
'diff_mem' => $current - $previous,
];
$previous = $current;
return $stats;
});Start LaravelS and request /debug-memory-leak until diff_mem is less than or equal to zero; if diff_mem is always greater than zero, it means that there may be a memory leak in Global Middleware or Laravel Framework;
After completing Step 3, alternately request the business routes and /debug-memory-leak (It is recommended to use ab/wrk to make a large number of requests for business routes), the initial increase in memory is normal. After a large number of requests for the business routes, if diff_mem is always greater than zero and curr_mem continues to increase, there is a high probability of memory leak; If curr_mem always changes within a certain range and does not continue to increase, there is a low probability of memory leak.
If you still can't solve it, max_request is the last guarantee.
Linux kernel parameter adjustment
Pressure test
Author: hhxsv5
Source Code: https://github.com/hhxsv5/laravel-s
License: MIT License