Socket: Non-blocking socket and TLS functionality for PHP based on Amp

Socket

amphp/socket is a socket library for establishing and encrypting non-blocking sockets PHP based on Amp.

Installation

This package can be installed as a Composer dependency.

composer require amphp/socket

Documentation

Documentation can be found on amphp.org as well as in the ./docs directory.

Examples

You can find more examples in the ./examples directory.

Client Example

<?php // basic (and dumb) HTTP client

require __DIR__ . '/../vendor/autoload.php';

// This is a very simple HTTP client that just prints the response without parsing.
// league/uri-schemes required for this example.

use Amp\ByteStream;
use Amp\Loop;
use Amp\Socket\ClientTlsContext;
use Amp\Socket\ConnectContext;
use Amp\Socket\EncryptableSocket;
use League\Uri;
use function Amp\Socket\connect;

Loop::run(static function () use ($argv) {
    $stdout = ByteStream\getStdout();

    if (\count($argv) !== 2) {
        yield $stdout->write('Usage: examples/simple-http-client.php <url>' . PHP_EOL);
        exit(1);
    }

    $uri = Uri\Http::createFromString($argv[1]);
    $host = $uri->getHost();
    $port = $uri->getPort() ?? ($uri->getScheme() === 'https' ? 443 : 80);
    $path = $uri->getPath() ?: '/';

    $connectContext = (new ConnectContext)
        ->withTlsContext(new ClientTlsContext($host));

    /** @var EncryptableSocket $socket */
    $socket = yield connect($host . ':' . $port, $connectContext);

    if ($uri->getScheme() === 'https') {
        yield $socket->setupTls();
    }

    yield $socket->write("GET {$path} HTTP/1.1\r\nHost: $host\r\nConnection: close\r\n\r\n");

    while (null !== $chunk = yield $socket->read()) {
        yield $stdout->write($chunk);
    }

    // If the promise returned from `read()` resolves to `null`, the socket closed and we're done.
    // In this case you can also use `yield Amp\ByteStream\pipe($socket, $stdout)` instead,
    // but we want to demonstrate the `read()` method here.
});

Server Example

<?php // basic (and dumb) HTTP server

require __DIR__ . '/../vendor/autoload.php';

// This is a very simple HTTP server that just prints a message to each client that connects.
// It doesn't check whether the client sent an HTTP request.

// You might notice that your browser opens several connections instead of just one,
// even when only making one request.

use Amp\Loop;
use Amp\Socket\ResourceSocket;
use Amp\Socket\Server;
use function Amp\asyncCoroutine;

Loop::run(static function () {
    $clientHandler = asyncCoroutine(static function (ResourceSocket $socket) {
        $address = $socket->getRemoteAddress();
        $ip = $address->getHost();
        $port = $address->getPort();

        echo "Accepted connection from {$address}." . PHP_EOL;

        $body = "Hey, your IP is {$ip} and your local port used is {$port}.";
        $bodyLength = \strlen($body);

        $req = "HTTP/1.1 200 OK\r\nConnection: close\r\nContent-Length: {$bodyLength}\r\n\r\n{$body}";
        yield $socket->end($req);
    });

    $server = Server::listen('127.0.0.1:0');

    echo 'Listening for new connections on ' . $server->getAddress() . ' ...' . PHP_EOL;
    echo 'Open your browser and visit http://' . $server->getAddress() . '/' . PHP_EOL;

    while ($socket = yield $server->accept()) {
        $clientHandler($socket);
    }
});

Security

If you discover any security related issues, please email me@kelunik.com instead of using the issue tracker.

Download Details:

Author: Amphp
Source Code: https://github.com/amphp/socket 
License: MIT license

#php #client #socket #unix 

What is GEEK

Buddha Community

A Collection Of Swift Tips & Tricks That I've Shared on Twitter

Swift tips & tricks ⚡️

One of the things I really love about Swift is how I keep finding interesting ways to use it in various situations, and when I do - I usually share them on Twitter. Here's a collection of all the tips & tricks that I've shared so far. Each entry has a link to the original tweet, if you want to respond with some feedback or question, which is always super welcome! 🚀

⚠️ This list is no longer being updated. For my latest Swift tips, checkout the "Tips" section on Swift by Sundell.

Also make sure to check out all of my other Swift content:

• 📖 My weekly articles, with a new post about Swift every Sunday!
• 🎧 My podcast which features special guests from around the community.
• 📬 My monthly newsletter - rounding up all of my content, including these tips, on the 1st of every month.

102 Making async tests faster and more stable

🚀 Here are some quick tips to make async tests faster & more stable:

• 😴 Avoid sleep() - use expectations instead
• ⏱ Use generous timeouts to avoid flakiness on CI
• 🧐 Put all assertions at the end of each test, not inside closures

// BEFORE:

class MentionDetectorTests: XCTestCase {
    func testDetectingMention() {
        let detector = MentionDetector()
        let string = "This test was written by @johnsundell."

        detector.detectMentions(in: string) { mentions in
            XCTAssertEqual(mentions, ["johnsundell"])
        }
        
        sleep(2)
    }
}

// AFTER:

class MentionDetectorTests: XCTestCase {
    func testDetectingMention() {
        let detector = MentionDetector()
        let string = "This test was written by @johnsundell."

        var mentions: [String]?
        let expectation = self.expectation(description: #function)

        detector.detectMentions(in: string) {
            mentions = $0
            expectation.fulfill()
        }

        waitForExpectations(timeout: 10)
        XCTAssertEqual(mentions, ["johnsundell"])
    }
}

For more on async testing, check out "Unit testing asynchronous Swift code".

101 Adding support for Apple Pencil double-taps

✍️ Adding support for the new Apple Pencil double-tap feature is super easy! All you have to do is to create a UIPencilInteraction, add it to a view, and implement one delegate method. Hopefully all pencil-compatible apps will soon adopt this.

let interaction = UIPencilInteraction()
interaction.delegate = self
view.addInteraction(interaction)

extension ViewController: UIPencilInteractionDelegate {
    func pencilInteractionDidTap(_ interaction: UIPencilInteraction) {
        // Handle pencil double-tap
    }
}

For more on using this and other iPad Pro features, check out "Building iPad Pro features in Swift".

100 Combining values with functions

😎 Here's a cool function that combines a value with a function to return a closure that captures that value, so that it can be called without any arguments. Super useful when working with closure-based APIs and we want to use some of our properties without having to capture self.

func combine<A, B>(_ value: A, with closure: @escaping (A) -> B) -> () -> B {
    return { closure(value) }
}

// BEFORE:

class ProductViewController: UIViewController {
    override func viewDidLoad() {
        super.viewDidLoad()

        buyButton.handler = { [weak self] in
            guard let self = self else {
                return
            }
            
            self.productManager.startCheckout(for: self.product)
        }
    }
}

// AFTER:

class ProductViewController: UIViewController {
    override func viewDidLoad() {
        super.viewDidLoad()

        buyButton.handler = combine(product, with: productManager.startCheckout)
    }
}

99 Dependency injection using functions

💉 When I'm only using a single function from a dependency, I love to inject that function as a closure, instead of having to create a protocol and inject the whole object. Makes dependency injection & testing super simple.

final class ArticleLoader {
    typealias Networking = (Endpoint) -> Future<Data>
    
    private let networking: Networking
    
    init(networking: @escaping Networking = URLSession.shared.load) {
        self.networking = networking
    }
    
    func loadLatest() -> Future<[Article]> {
        return networking(.latestArticles).decode()
    }
}

For more on this technique, check out "Simple Swift dependency injection with functions".

98 Using a custom exception handler

💥 It's cool that you can easily assign a closure as a custom NSException handler. This is super useful when building things in Playgrounds - since you can't use breakpoints - so instead of just signal SIGABRT, you'll get the full exception description if something goes wrong.

NSSetUncaughtExceptionHandler { exception in
    print(exception)
}

97 Using type aliases to give semantic meaning to primitives

❤️ I love that in Swift, we can use the type system to make our code so much more self-documenting - one way of doing so is to use type aliases to give the primitive types that we use a more semantic meaning.

extension List.Item {
    // Using type aliases, we can give semantic meaning to the
    // primitive types that we use, without having to introduce
    // wrapper types.
    typealias Index = Int
}

extension List {
    enum Mutation {
        // Our enum cases now become a lot more self-documenting,
        // without having to add additional parameter labels to
        // explain them.
        case add(Item, Item.Index)
        case update(Item, Item.Index)
        case remove(Item.Index)
    }
}

For more on self-documenting code, check out "Writing self-documenting Swift code".

96 Specializing protocols using constraints

🤯 A little late night prototyping session reveals that protocol constraints can not only be applied to extensions - they can also be added to protocol definitions!

This is awesome, since it lets us easily define specialized protocols based on more generic ones.

protocol Component {
    associatedtype Container
    func add(to container: Container)
}

// Protocols that inherit from other protocols can include
// constraints to further specialize them.
protocol ViewComponent: Component where Container == UIView {
    associatedtype View: UIView
    var view: View { get }
}

extension ViewComponent {
    func add(to container: UIView) {
        container.addSubview(view)
    }
}

For more on specializing protocols, check out "Specializing protocols in Swift".

95 Unwrapping an optional or throwing an error

📦 Here's a super handy extension on Swift's Optional type, which gives us a really nice API for easily unwrapping an optional, or throwing an error in case the value turned out to be nil:

extension Optional {
    func orThrow(_ errorExpression: @autoclosure () -> Error) throws -> Wrapped {
        switch self {
        case .some(let value):
            return value
        case .none:
            throw errorExpression()
        }
    }
}

let file = try loadFile(at: path).orThrow(MissingFileError())

For more ways that optionals can be extended, check out "Extending optionals in Swift".

94 Testing code that uses static APIs

👩‍🔬 Testing code that uses static APIs can be really tricky, but there's a way that it can often be done - using Swift's first class function capabilities!

Instead of accessing that static API directly, we can inject the function we want to use, which enables us to mock it!

// BEFORE

class FriendsLoader {
    func loadFriends(then handler: @escaping (Result<[Friend]>) -> Void) {
        Networking.loadData(from: .friends) { result in
            ...
        }
    }
}

// AFTER

class FriendsLoader {
    typealias Handler<T> = (Result<T>) -> Void
    typealias DataLoadingFunction = (Endpoint, @escaping Handler<Data>) -> Void

    func loadFriends(using dataLoading: DataLoadingFunction = Networking.loadData,
                     then handler: @escaping Handler<[Friend]>) {
        dataLoading(.friends) { result in
            ...
        }
    }
}

// MOCKING IN TESTS

let dataLoading: FriendsLoader.DataLoadingFunction = { _, handler in
    handler(.success(mockData))
}

friendsLoader.loadFriends(using: dataLoading) { result in
    ...
}

93 Matching multiple enum cases with associated values

🐾 Swift's pattern matching capabilities are so powerful! Two enum cases with associated values can even be matched and handled by the same switch case - which is super useful when handling state changes with similar data.

enum DownloadState {
    case inProgress(progress: Double)
    case paused(progress: Double)
    case cancelled
    case finished(Data)
}

func downloadStateDidChange(to state: DownloadState) {
    switch state {
    case .inProgress(let progress), .paused(let progress):
        updateProgressView(with: progress)
    case .cancelled:
        showCancelledMessage()
    case .finished(let data):
        process(data)
    }
}

92 Multiline string literals

🅰 One really nice benefit of Swift multiline string literals - even for single lines of text - is that they don't require quotes to be escaped. Perfect when working with things like HTML, or creating a custom description for an object.

let html = highlighter.highlight("Array<String>")

XCTAssertEqual(html, """
<span class="type">Array</span>&lt;<span class="type">String</span>&gt;
""")

91 Reducing sequences

💎 While it's very common in functional programming, the reduce function might be a bit of a hidden gem in Swift. It provides a super useful way to transform a sequence into a single value.

extension Sequence where Element: Equatable {
    func numberOfOccurrences(of target: Element) -> Int {
        return reduce(0) { result, element in
            guard element == target else {
                return result
            }

            return result + 1
        }
    }
}

You can read more about transforming collections in "Transforming collections in Swift".

90 Avoiding manual Codable implementations

📦 When I use Codable in Swift, I want to avoid manual implementations as much as possible, even when there's a mismatch between my code structure and the JSON I'm decoding.

One way that can often be achieved is to use private data containers combined with computed properties.

struct User: Codable {
    let name: String
    let age: Int

    var homeTown: String { return originPlace.name }

    private let originPlace: Place
}

private extension User {
    struct Place: Codable {
        let name: String
    }
}

extension User {
    struct Container: Codable {
        let user: User
    }
}

89 Using feature flags instead of feature branches

🚢 Instead of using feature branches, I merge almost all of my code directly into master - and then I use feature flags to conditionally enable features when they're ready. That way I can avoid merge conflicts and keep shipping!

extension ListViewController {
    func addSearchIfNeeded() {
        // Rather than having to keep maintaining a separate
        // feature branch for a new feature, we can use a flag
        // to conditionally turn it on.
        guard FeatureFlags.searchEnabled else {
            return
        }

        let resultsVC = SearchResultsViewController()
        let searchVC = UISearchController(
            searchResultsController: resultsVC
        )

        searchVC.searchResultsUpdater = resultsVC
        navigationItem.searchController = searchVC
    }
}

You can read more about feature flags in "Feature flags in Swift".

88 Lightweight data hierarchies using tuples

💾 Here I'm using tuples to create a lightweight hierarchy for my data, giving me a nice structure without having to introduce any additional types.

struct CodeSegment {
    var tokens: (
        previous: String?,
        current: String
    )

    var delimiters: (
        previous: Character?
        next: Character?
    )
}

handle(segment.tokens.current)

You can read more about tuples in "Using tuples as lightweight types in Swift"

87 The rule of threes

3️⃣ Whenever I have 3 properties or local variables that share the same prefix, I usually try to extract them into their own method or type. That way I can avoid massive types & methods, and also increase readability, without falling into a "premature optimization" trap.

Before

public func generate() throws {
    let contentFolder = try folder.subfolder(named: "content")

    let articleFolder = try contentFolder.subfolder(named: "posts")
    let articleProcessor = ContentProcessor(folder: articleFolder)
    let articles = try articleProcessor.process()

    ...
}

After

public func generate() throws {
    let contentFolder = try folder.subfolder(named: "content")
    let articles = try processArticles(in: contentFolder)
    ...
}

private func processArticles(in folder: Folder) throws -> [ContentItem] {
    let folder = try folder.subfolder(named: "posts")
    let processor = ContentProcessor(folder: folder)
    return try processor.process()
}

86 Useful Codable extensions

👨‍🔧 Here's two extensions that I always add to the Encodable & Decodable protocols, which for me really make the Codable API nicer to use. By using type inference for decoding, a lot of boilerplate can be removed when the compiler is already able to infer the resulting type.

extension Encodable {
    func encoded() throws -> Data {
        return try JSONEncoder().encode(self)
    }
}

extension Data {
    func decoded<T: Decodable>() throws -> T {
        return try JSONDecoder().decode(T.self, from: self)
    }
}

let data = try user.encoded()

// By using a generic type in the decoded() method, the
// compiler can often infer the type we want to decode
// from the current context.
try userDidLogin(data.decoded())

// And if not, we can always supply the type, still making
// the call site read very nicely.
let otherUser = try data.decoded() as User

85 Using shared UserDefaults suites

📦 UserDefaults is a lot more powerful than what it first might seem like. Not only can it store more complex values (like dates & dictionaries) and parse command line arguments - it also enables easy sharing of settings & lightweight data between apps in the same App Group.

let sharedDefaults = UserDefaults(suiteName: "my-app-group")!
let useDarkMode = sharedDefaults.bool(forKey: "dark-mode")

// This value is put into the shared suite.
sharedDefaults.set(true, forKey: "dark-mode")

// If you want to treat the shared settings as read-only (and add
// local overrides on top of them), you can simply add the shared
// suite to the standard UserDefaults.
let combinedDefaults = UserDefaults.standard
combinedDefaults.addSuite(named: "my-app-group")

// This value is a local override, not added to the shared suite.
combinedDefaults.set(true, forKey: "app-specific-override")

84 Custom UIView backing layers

🎨 By overriding layerClass you can tell UIKit what CALayer class to use for a UIView's backing layer. That way you can reduce the amount of layers, and don't have to do any manual layout.

final class GradientView: UIView {
    override class var layerClass: AnyClass { return CAGradientLayer.self }

    var colors: (start: UIColor, end: UIColor)? {
        didSet { updateLayer() }
    }

    private func updateLayer() {
        let layer = self.layer as! CAGradientLayer
        layer.colors = colors.map { [$0.start.cgColor, $0.end.cgColor] }
    }
}

83 Auto-Equatable enums with associated values

✅ That the compiler now automatically synthesizes Equatable conformances is such a huge upgrade for Swift! And the cool thing is that it works for all kinds of types - even for enums with associated values! Especially useful when using enums for verification in unit tests.

struct Article: Equatable {
    let title: String
    let text: String
}

struct User: Equatable {
    let name: String
    let age: Int
}

extension Navigator {
    enum Destination: Equatable {
        case profile(User)
        case article(Article)
    }
}

func testNavigatingToArticle() {
    let article = Article(title: "Title", text: "Text")
    controller.select(article)
    XCTAssertEqual(navigator.destinations, [.article(article)])
}

82 Defaults for associated types

🤝 Associated types can have defaults in Swift - which is super useful for types that are not easily inferred (for example when they're not used for a specific instance method or property).

protocol Identifiable {
    associatedtype RawIdentifier: Codable = String

    var id: Identifier<Self> { get }
}

struct User: Identifiable {
    let id: Identifier<User>
    let name: String
}

struct Group: Identifiable {
    typealias RawIdentifier = Int

    let id: Identifier<Group>
    let name: String
}

81 Creating a dedicated identifier type

🆔 If you want to avoid using plain strings as identifiers (which can increase both type safety & readability), it's really easy to create a custom Identifier type that feels just like a native Swift type, thanks to protocols!

More on this topic in "Type-safe identifiers in Swift".

struct Identifier: Hashable {
    let string: String
}

extension Identifier: ExpressibleByStringLiteral {
    init(stringLiteral value: String) {
        string = value
    }
}

extension Identifier: CustomStringConvertible {
    var description: String {
        return string
    }
}

extension Identifier: Codable {
    init(from decoder: Decoder) throws {
        let container = try decoder.singleValueContainer()
        string = try container.decode(String.self)
    }

    func encode(to encoder: Encoder) throws {
        var container = encoder.singleValueContainer()
        try container.encode(string)
    }
}

struct Article: Codable {
    let id: Identifier
    let title: String
}

let article = Article(id: "my-article", title: "Hello world!")

80 Assigning optional tuple members to variables

🙌 A really cool thing about using tuples to model the internal state of a Swift type, is that you can unwrap an optional tuple's members directly into local variables.

Very useful in order to group multiple optional values together for easy unwrapping & handling.

class ImageTransformer {
    private var queue = [(image: UIImage, transform: Transform)]()

    private func processNext() {
        // When unwrapping an optional tuple, you can assign the members
        // directly to local variables.
        guard let (image, transform) = queue.first else {
            return
        }

        let context = Context()
        context.draw(image)
        context.apply(transform)
        ...
    }
}

79 Struct convenience initializers

❤️ I love to structure my code using extensions in Swift. One big benefit of doing so when it comes to struct initializers, is that defining a convenience initializer doesn't remove the default one the compiler generates - best of both worlds!

struct Article {
    let date: Date
    var title: String
    var text: String
    var comments: [Comment]
}

extension Article {
    init(title: String, text: String) {
        self.init(date: Date(), title: title, text: text, comments: [])
    }
}

let articleA = Article(title: "Best Cupcake Recipe", text: "...")

let articleB = Article(
    date: Date(),
    title: "Best Cupcake Recipe",
    text: "...",
    comments: [
        Comment(user: currentUser, text: "Yep, can confirm!")
    ]
)

78 Usages of throwing functions

🏈 A big benefit of using throwing functions for synchronous Swift APIs is that the caller can decide whether they want to treat the return value as optional (try?) or required (try).

func loadFile(named name: String) throws -> File {
    guard let url = urlForFile(named: name) else {
        throw File.Error.missing
    }

    do {
        let data = try Data(contentsOf: url)
        return File(url: url, data: data)
    } catch {
        throw File.Error.invalidData(error)
    }
}

let requiredFile = try loadFile(named: "AppConfig.json")

let optionalFile = try? loadFile(named: "UserSettings.json")

77 Nested generic types

🐝 Types that are nested in generics automatically inherit their parent's generic types - which is super useful when defining accessory types (for things like states or outcomes).

struct Task<Input, Output> {
    typealias Closure = (Input) throws -> Output

    let closure: Closure
}

extension Task {
    enum Result {
        case success(Output)
        case failure(Error)
    }
}

76 Equatable & Hashable structures

🤖 Now that the Swift compiler automatically synthesizes Equatable & Hashable conformances for value types, it's easier than ever to setup model structures with nested types that are all Equatable/Hashable!

typealias Value = Hashable & Codable

struct User: Value {
    var name: String
    var age: Int
    var lastLoginDate: Date?
    var settings: Settings
}

extension User {
    struct Settings: Value {
        var itemsPerPage: Int
        var theme: Theme
    }
}

extension User.Settings {
    enum Theme: String, Value {
        case light
        case dark
    }
}

You can read more about using nested types in Swift here.

75 Conditional conformances

🎉 Swift 4.1 is here! One of the key features it brings is conditional conformances, which lets you have a type only conform to a protocol under certain constraints.

protocol UnboxTransformable {
    associatedtype RawValue

    static func transform(_ value: RawValue) throws -> Self?
}

extension Array: UnboxTransformable where Element: UnboxTransformable {
    typealias RawValue = [Element.RawValue]

    static func transform(_ value: RawValue) throws -> [Element]? {
        return try value.compactMap(Element.transform)
    }
}

I also have an article with lots of more info on conditional conformances here. Paul Hudson also has a great overview of all Swift 4.1 features here.

74 Generic type aliases

🕵️‍♀️ A cool thing about Swift type aliases is that they can be generic! Combine that with tuples and you can easily define simple generic types.

typealias Pair<T> = (T, T)

extension Game {
    func calculateScore(for players: Pair<Player>) -> Int {
        ...
    }
}

You can read more about using tuples as lightweight types here.

73 Parsing command line arguments using UserDefaults

☑️ A really cool "hidden" feature of UserDefaults is that it contains any arguments that were passed to the app at launch!

Super useful both in Swift command line tools & scripts, but also to temporarily override a value when debugging iOS apps.

let defaults = UserDefaults.standard
let query = defaults.string(forKey: "query")
let resultCount = defaults.integer(forKey: "results")

72 Using the & operator

👏 Swift's & operator is awesome! Not only can you use it to compose protocols, you can compose other types too! Very useful if you want to hide concrete types & implementation details.

protocol LoadableFromURL {
    func load(from url: URL)
}

class ContentViewController: UIViewController, LoadableFromURL {
    func load(from url: URL) {
        ...
    }
}

class ViewControllerFactory {
    func makeContentViewController() -> UIViewController & LoadableFromURL {
        return ContentViewController()
    }
}

71 Capturing multiple values in mocks

🤗 When capturing values in mocks, using an array (instead of just a single value) makes it easy to verify that only a certain number of values were passed.

Perfect for protecting against "over-calling" something.

class UserManagerTests: XCTestCase {
    func testObserversCalledWhenUserFirstLogsIn() {
        let manager = UserManager()

        let observer = ObserverMock()
        manager.addObserver(observer)

        // First login, observers should be notified
        let user = User(id: 123, name: "John")
        manager.userDidLogin(user)
        XCTAssertEqual(observer.users, [user])

        // If the same user logs in again, observers shouldn't be notified
        manager.userDidLogin(user)
        XCTAssertEqual(observer.users, [user])
    }
}

private extension UserManagerTests {
    class ObserverMock: UserManagerObserver {
        private(set) var users = [User]()

        func userDidChange(to user: User) {
            users.append(user)
        }
    }
}

70 Reducing the need for mocks

👋 When writing tests, you don't always need to create mocks - you can create stubs using real instances of things like errors, URLs & UserDefaults.

Here's how to do that for some common tasks/object types in Swift:

// Create errors using NSError (#function can be used to reference the name of the test)
let error = NSError(domain: #function, code: 1, userInfo: nil)

// Create non-optional URLs using file paths
let url = URL(fileURLWithPath: "Some/URL")

// Reference the test bundle using Bundle(for:)
let bundle = Bundle(for: type(of: self))

// Create an explicit UserDefaults object (instead of having to use a mock)
let userDefaults = UserDefaults(suiteName: #function)

// Create queues to control/await concurrent operations
let queue = DispatchQueue(label: #function)

For when you actually do need mocking, check out "Mocking in Swift".

69 Using "then" as an external parameter label for closures

⏱ I've started using "then" as an external parameter label for completion handlers. Makes the call site read really nicely (Because I do ❤️ conversational API design) regardless of whether trailing closure syntax is used or not.

protocol DataLoader {
    // Adding type aliases to protocols can be a great way to
    // reduce verbosity for parameter types.
    typealias Handler = (Result<Data>) -> Void
    associatedtype Endpoint

    func loadData(from endpoint: Endpoint, then handler: @escaping Handler)
}

loader.loadData(from: .messages) { result in
    ...
}

loader.loadData(from: .messages, then: { result in
    ...
})

68 Combining lazily evaluated sequences with the builder pattern

😴 Combining lazily evaluated sequences with builder pattern-like properties can lead to some pretty sweet APIs for configurable sequences in Swift.

Also useful for queries & other things you "build up" and then execute.

// Extension adding builder pattern-like properties that return
// a new sequence value with the given configuration applied
extension FileSequence {
    var recursive: FileSequence {
        var sequence = self
        sequence.isRecursive = true
        return sequence
    }

    var includingHidden: FileSequence {
        var sequence = self
        sequence.includeHidden = true
        return sequence
    }
}

// BEFORE

let files = folder.makeFileSequence(recursive: true, includeHidden: true)

// AFTER

let files = folder.files.recursive.includingHidden

Want an intro to lazy sequences? Check out "Swift sequences: The art of being lazy".

67 Faster & more stable UI tests

My top 3 tips for faster & more stable UI tests:

📱 Reset the app's state at the beginning of every test.

🆔 Use accessibility identifiers instead of UI strings.

⏱ Use expectations instead of waiting time.

func testOpeningArticle() {
    // Launch the app with an argument that tells it to reset its state
    let app = XCUIApplication()
    app.launchArguments.append("--uitesting")
    app.launch()
    
    // Check that the app is displaying an activity indicator
    let activityIndicator = app.activityIndicator.element
    XCTAssertTrue(activityIndicator.exists)
    
    // Wait for the loading indicator to disappear = content is ready
    expectation(for: NSPredicate(format: "exists == 0"),
                evaluatedWith: activityIndicator)
                
    // Use a generous timeout in case the network is slow
    waitForExpectations(timeout: 10)
    
    // Tap the cell for the first article
    app.tables.cells["Article.0"].tap()
    
    // Assert that a label with the accessibility identifier "Article.Title" exists
    let label = app.staticTexts["Article.Title"]
    XCTAssertTrue(label.exists)
}

66 Accessing the clipboard from a Swift script

📋 It's super easy to access the contents of the clipboard from a Swift script. A big benefit of Swift scripting is being able to use Cocoa's powerful APIs for Mac apps.

import Cocoa

let clipboard = NSPasteboard.general.string(forType: .string)

65 Using tuples for view state

🎯 Using Swift tuples for view state can be a super nice way to group multiple properties together and render them reactively using the layout system.

By using a tuple we don't have to either introduce a new type or make our view model-aware.

class TextView: UIView {
    var state: (title: String?, text: String?) {
        // By telling UIKit that our view needs layout and binding our
        // state in layoutSubviews, we can react to state changes without
        // doing unnecessary layout work.
        didSet { setNeedsLayout() }
    }

    private let titleLabel = UILabel()
    private let textLabel = UILabel()

    override func layoutSubviews() {
        super.layoutSubviews()

        titleLabel.text = state.title
        textLabel.text = state.text

        ...
    }
}

64 Throwing tests and LocalizedError

⚾️ Swift tests can throw, which is super useful in order to avoid complicated logic or force unwrapping. By making errors conform to LocalizedError, you can also get a nice error message in Xcode if there's a failure.

class ImageCacheTests: XCTestCase {
    func testCachingAndLoadingImage() throws {
        let bundle = Bundle(for: type(of: self))
        let cache = ImageCache(bundle: bundle)
        
        // Bonus tip: You can easily load images from your test
        // bundle using this UIImage initializer
        let image = try require(UIImage(named: "sample", in: bundle, compatibleWith: nil))
        try cache.cache(image, forKey: "key")
        
        let cachedImage = try cache.image(forKey: "key")
        XCTAssertEqual(image, cachedImage)
    }
}

enum ImageCacheError {
    case emptyKey
    case dataConversionFailed
}

// When using throwing tests, making your errors conform to
// LocalizedError will render a much nicer error message in
// Xcode (per default only the error code is shown).
extension ImageCacheError: LocalizedError {
    var errorDescription: String? {
        switch self {
        case .emptyKey:
            return "An empty key was given"
        case .dataConversionFailed:
            return "Failed to convert the given image to Data"
        }
    }
}

For more information, and the implementation of the require method used above, check out "Avoiding force unwrapping in Swift unit tests".

63 The difference between static and class properties

✍️ Unlike static properties, class properties can be overridden by subclasses (however, they can't be stored, only computed).

class TableViewCell: UITableViewCell {
    class var preferredHeight: CGFloat { return 60 }
}

class TallTableViewCell: TableViewCell {
    override class var preferredHeight: CGFloat { return 100 }
}

62 Creating extensions with static factory methods

👨‍🎨 Creating extensions with static factory methods can be a great alternative to subclassing in Swift, especially for things like setting up UIViews, CALayers or other kinds of styling.

It also lets you remove a lot of styling & setup from your view controllers.

extension UILabel {
    static func makeForTitle() -> UILabel {
        let label = UILabel()
        label.font = .boldSystemFont(ofSize: 24)
        label.textColor = .darkGray
        label.adjustsFontSizeToFitWidth = true
        label.minimumScaleFactor = 0.75
        return label
    }

    static func makeForText() -> UILabel {
        let label = UILabel()
        label.font = .systemFont(ofSize: 16)
        label.textColor = .black
        label.numberOfLines = 0
        return label
    }
}

class ArticleViewController: UIViewController {
    lazy var titleLabel = UILabel.makeForTitle()
    lazy var textLabel = UILabel.makeForText()
}

61 Child view controller auto-resizing

🧒 An awesome thing about child view controllers is that they're automatically resized to match their parent, making them a super nice solution for things like loading & error views.

class ListViewController: UIViewController {
    func loadItems() {
        let loadingViewController = LoadingViewController()
        add(loadingViewController)

        dataLoader.loadItems { [weak self] result in
            loadingViewController.remove()
            self?.handle(result)
        }
    }
}

For more about child view controller (including the add and remove methods used above), check out "Using child view controllers as plugins in Swift".

60 Using zip

🤐 Using the zip function in Swift you can easily combine two sequences. Super useful when using two sequences to do some work, since zip takes care of all the bounds-checking.

func render(titles: [String]) {
    for (label, text) in zip(titleLabels, titles) {
        print(text)
        label.text = text
    }
}

59 Defining custom option sets

🎛 The awesome thing about option sets in Swift is that they can automatically either be passed as a single member or as a set. Even cooler is that you can easily define your own option sets as well, perfect for options and other non-exclusive values.

// Option sets are awesome, because you can easily pass them
// both using dot syntax and array literal syntax, like when
// using the UIView animation API:
UIView.animate(withDuration: 0.3,
               delay: 0,
               options: .allowUserInteraction,
               animations: animations)

UIView.animate(withDuration: 0.3,
               delay: 0,
               options: [.allowUserInteraction, .layoutSubviews],
               animations: animations)

// The cool thing is that you can easily define your own option
// sets as well, by defining a struct that has an Int rawValue,
// that will be used as a bit mask.
extension Cache {
    struct Options: OptionSet {
        static let saveToDisk = Options(rawValue: 1)
        static let clearOnMemoryWarning = Options(rawValue: 1 << 1)
        static let clearDaily = Options(rawValue: 1 << 2)

        let rawValue: Int
    }
}

// We can now use Cache.Options just like UIViewAnimationOptions:
Cache(options: .saveToDisk)
Cache(options: [.saveToDisk, .clearDaily])

58 Using the where clause with associated types

🙌 Using the where clause when designing protocol-oriented APIs in Swift can let your implementations (or others' if it's open source) have a lot more freedom, especially when it comes to collections.

See "Using generic type constraints in Swift 4" for more info.

public protocol PathFinderMap {
    associatedtype Node
    // Using the 'where' clause for associated types, we can
    // ensure that a type meets certain requirements (in this
    // case that it's a sequence with Node elements).
    associatedtype NodeSequence: Sequence where NodeSequence.Element == Node

    // Instead of using a concrete type (like [Node]) here, we
    // give implementors of this protocol more freedom while
    // still meeting our requirements. For example, one
    // implementation might use Set<Node>.
    func neighbors(of node: Node) -> NodeSequence
}

57 Using first class functions when iterating over a dictionary

👨‍🍳 Combine first class functions in Swift with the fact that Dictionary elements are (Key, Value) tuples and you can build yourself some pretty awesome functional chains when iterating over a Dictionary.

func makeActor(at coordinate: Coordinate, for building: Building) -> Actor {
    let actor = Actor()
    actor.position = coordinate.point
    actor.animation = building.animation
    return actor
}

func render(_ buildings: [Coordinate : Building]) {
    buildings.map(makeActor).forEach(add)
}

56 Calling instance methods as static functions

😎 In Swift, you can call any instance method as a static function and it will return a closure representing that method. This is how running tests using SPM on Linux works.

More about this topic in my blog post "First class functions in Swift".

// This produces a '() -> Void' closure which is a reference to the
// given view's 'removeFromSuperview' method.
let closure = UIView.removeFromSuperview(view)

// We can now call it just like we would any other closure, and it
// will run 'view.removeFromSuperview()'
closure()

// This is how running tests using the Swift Package Manager on Linux
// works, you return your test functions as closures:
extension UserManagerTests {
    static var allTests = [
        ("testLoggingIn", testLoggingIn),
        ("testLoggingOut", testLoggingOut),
        ("testUserPermissions", testUserPermissions)
    ]
}

55 Dropping suffixes from method names to support multiple arguments

👏 One really nice benefit of dropping suffixes from method names (and just using verbs, when possible) is that it becomes super easy to support both single and multiple arguments, and it works really well semantically.

extension UIView {
    func add(_ subviews: UIView...) {
        subviews.forEach(addSubview)
    }
}

view.add(button)
view.add(label)

// By dropping the "Subview" suffix from the method name, both
// single and multiple arguments work really well semantically.
view.add(button, label)

54 Constraining protocols to classes to ensure mutability

👽 Using the AnyObject (or class) constraint on protocols is not only useful when defining delegates (or other weak references), but also when you always want instances to be mutable without copying.

// By constraining a protocol with 'AnyObject' it can only be adopted
// by classes, which means all instances will always be mutable, and
// that it's the original instance (not a copy) that will be mutated.
protocol DataContainer: AnyObject {
    var data: Data? { get set }
}

class UserSettingsManager {
    private var settings: Settings
    private let dataContainer: DataContainer

    // Since DataContainer is a protocol, we an easily mock it in
    // tests if we use dependency injection
    init(settings: Settings, dataContainer: DataContainer) {
        self.settings = settings
        self.dataContainer = dataContainer
    }

    func saveSettings() throws {
        let data = try settings.serialize()

        // We can now assign properties on an instance of our protocol
        // because the compiler knows it's always going to be a class
        dataContainer.data = data
    }
}

53 String-based enums in string interpolation

🍣 Even if you define a custom raw value for a string-based enum in Swift, the full case name will be used in string interpolation.

Super useful when using separate raw values for JSON, while still wanting to use the full case name in other contexts.

extension Building {
    // This enum has custom raw values that are used when decoding
    // a value, for example from JSON.
    enum Kind: String {
        case castle = "C"
        case town = "T"
        case barracks = "B"
        case goldMine = "G"
        case camp = "CA"
        case blacksmith = "BL"
    }

    var animation: Animation {
        return Animation(
            // When used in string interpolation, the full case name is still used.
            // For 'castle' this will be 'buildings/castle'.
            name: "buildings/\(kind)",
            frameCount: frameCount,
            frameDuration: frameDuration
        )
    }
}

52 Expressively comparing a value with a list of candidates

👨‍🔬 Continuing to experiment with expressive ways of comparing a value with a list of candidates in Swift. Adding an extension on Equatable is probably my favorite approach so far.

extension Equatable {
    func isAny(of candidates: Self...) -> Bool {
        return candidates.contains(self)
    }
}

let isHorizontal = direction.isAny(of: .left, .right)

See tip 35 for my previous experiment.

51 UIView bounds and transforms

📐 A really interesting side-effect of a UIView's bounds being its rect within its own coordinate system is that transforms don't affect it at all. That's why it's usually a better fit than frame when doing layout calculations of subviews.

let view = UIView()
view.frame.size = CGSize(width: 100, height: 100)
view.transform = CGAffineTransform(scaleX: 2, y: 2)

print(view.frame) // (-50.0, -50.0, 200.0, 200.0)
print(view.bounds) // (0.0, 0.0, 100.0, 100.0)

50 UIKit default arguments

👏 It's awesome that many UIKit APIs with completion handlers and other optional parameters import into Swift with default arguments (even though they are written in Objective-C). Getting rid of all those nil arguments is so nice!

// BEFORE: All parameters are specified, just like in Objective-C

viewController.present(modalViewController, animated: true, completion: nil)

modalViewController.dismiss(animated: true, completion: nil)

viewController.transition(from: loadingViewController,
                          to: contentViewController,
                          duration: 0.3,
                          options: [],
                          animations: animations,
                          completion: nil)

// AFTER: Since many UIKit APIs with completion handlers and other
// optional parameters import into Swift with default arguments,
// we can make our calls shorter

viewController.present(modalViewController, animated: true)

modalViewController.dismiss(animated: true)

viewController.transition(from: loadingViewController,
                          to: contentViewController,
                          duration: 0.3,
                          animations: animations)

49 Avoiding Massive View Controllers

✂️ Avoiding Massive View Controllers is all about finding the right levels of abstraction and splitting things up.

My personal rule of thumb is that as soon as I have 3 methods or properties that have the same prefix, I break them out into their own type.

// BEFORE

class LoginViewController: UIViewController {
    private lazy var signUpLabel = UILabel()
    private lazy var signUpImageView = UIImageView()
    private lazy var signUpButton = UIButton()
}

// AFTER

class LoginViewController: UIViewController {
    private lazy var signUpView = SignUpView()
}

class SignUpView: UIView {
    private lazy var label = UILabel()
    private lazy var imageView = UIImageView()
    private lazy var button = UIButton()
}

48 Extending optionals

❤️ I love the fact that optionals are enums in Swift - it makes it so easy to extend them with convenience APIs for certain types. Especially useful when doing things like data validation on optional values.

func validateTextFields() -> Bool {
    guard !usernameTextField.text.isNilOrEmpty else {
        return false
    }

    ...

    return true
}

// Since all optionals are actual enum values in Swift, we can easily
// extend them for certain types, to add our own convenience APIs

extension Optional where Wrapped == String {
    var isNilOrEmpty: Bool {
        switch self {
        case let string?:
            return string.isEmpty
        case nil:
            return true
        }
    }
}

// Since strings are now Collections in Swift 4, you can even
// add this property to all optional collections:

extension Optional where Wrapped: Collection {
    var isNilOrEmpty: Bool {
        switch self {
        case let collection?:
            return collection.isEmpty
        case nil:
            return true
        }
    }
}

47 Using where with for-loops

🗺 Using the where keyword can be a super nice way to quickly apply a filter in a for-loop in Swift. You can of course use map, filter and forEach, or guard, but for simple loops I think this is very expressive and nice.

func archiveMarkedPosts() {
    for post in posts where post.isMarked {
        archive(post)
    }
}

func healAllies() {
    for player in players where player.isAllied(to: currentPlayer) {
        player.heal()
    }
}

46 Variable shadowing

👻 Variable shadowing can be super useful in Swift, especially when you want to create a local copy of a parameter value in order to use it as state within a closure.

init(repeatMode: RepeatMode, closure: @escaping () -> UpdateOutcome) {
    // Shadow the argument with a local, mutable copy
    var repeatMode = repeatMode
    
    self.closure = {
        // With shadowing, there's no risk of accidentially
        // referring to the immutable version
        switch repeatMode {
        case .forever:
            break
        case .times(let count):
            guard count > 0 else {
                return .finished
            }
            
            // We can now capture the mutable version and use
            // it for state in a closure
            repeatMode = .times(count - 1)
        }
        
        return closure()
    }
}

45 Using dot syntax for static properties and initializers

✒️ Dot syntax is one of my favorite features of Swift. What's really cool is that it's not only for enums, any static method or property can be used with dot syntax - even initializers! Perfect for convenience APIs and default parameters.

public enum RepeatMode {
    case times(Int)
    case forever
}

public extension RepeatMode {
    static var never: RepeatMode {
        return .times(0)
    }

    static var once: RepeatMode {
        return .times(1)
    }
}

view.perform(animation, repeated: .once)

// To make default parameters more compact, you can even use init with dot syntax

class ImageLoader {
    init(cache: Cache = .init(), decoder: ImageDecoder = .init()) {
        ...
    }
}

44 Calling functions as closures with a tuple as parameters

🚀 One really cool aspect of Swift having first class functions is that you can pass any function (or even initializer) as a closure, and even call it with a tuple containing its parameters!

// This function lets us treat any "normal" function or method as
// a closure and run it with a tuple that contains its parameters
func call<Input, Output>(_ function: (Input) -> Output, with input: Input) -> Output {
    return function(input)
}

class ViewFactory {
    func makeHeaderView() -> HeaderView {
        // We can now pass an initializer as a closure, and a tuple
        // containing its parameters
        return call(HeaderView.init, with: loadTextStyles())
    }
    
    private func loadTextStyles() -> (font: UIFont, color: UIColor) {
        return (theme.font, theme.textColor)
    }
}

class HeaderView {
    init(font: UIFont, textColor: UIColor) {
        ...
    }
}

43 Enabling static dependency injection

💉 If you've been struggling to test code that uses static APIs, here's a technique you can use to enable static dependency injection without having to modify any call sites:

// Before: Almost impossible to test due to the use of singletons

class Analytics {
    static func log(_ event: Event) {
        Database.shared.save(event)
        
        let dictionary = event.serialize()
        NetworkManager.shared.post(dictionary, to: eventURL)
    }
}

// After: Much easier to test, since we can inject mocks as arguments

class Analytics {
    static func log(_ event: Event,
                    database: Database = .shared,
                    networkManager: NetworkManager = .shared) {
        database.save(event)
        
        let dictionary = event.serialize()
        networkManager.post(dictionary, to: eventURL)
    }
}

42 Type inference for lazy properties in Swift 4

🎉 In Swift 4, type inference works for lazy properties and you don't need to explicitly refer to self!

// Swift 3

class PurchaseView: UIView {
    private lazy var buyButton: UIButton = self.makeBuyButton()
    
    private func makeBuyButton() -> UIButton {
        let button = UIButton()
        button.setTitle("Buy", for: .normal)
        button.setTitleColor(.blue, for: .normal)
        return button
    }
}

// Swift 4

class PurchaseView: UIView {
    private lazy var buyButton = makeBuyButton()
    
    private func makeBuyButton() -> UIButton {
        let button = UIButton()
        button.setTitle("Buy", for: .normal)
        button.setTitleColor(.blue, for: .normal)
        return button
    }
}

41 Converting Swift errors to NSError

😎 You can turn any Swift Error into an NSError, which is super useful when pattern matching with a code 👍. Also, switching on optionals is pretty cool!

let task = urlSession.dataTask(with: url) { data, _, error in
    switch error {
    case .some(let error as NSError) where error.code == NSURLErrorNotConnectedToInternet:
        presenter.showOfflineView()
    case .some(let error):
        presenter.showGenericErrorView()
    case .none:
        presenter.renderContent(from: data)
    }
}

task.resume()

Also make sure to check out Kostas Kremizas' tip about how you can pattern match directly against a member of URLError.

40 Making UIImage macOS compatible

🖥 Here's an easy way to make iOS model code that uses UIImage macOS compatible - like me and Gui Rambo discussed on the Swift by Sundell Podcast.

// Either put this in a separate file that you only include in your macOS target or wrap the code in #if os(macOS) / #endif

import Cocoa

// Step 1: Typealias UIImage to NSImage
typealias UIImage = NSImage

// Step 2: You might want to add these APIs that UIImage has but NSImage doesn't.
extension NSImage {
    var cgImage: CGImage? {
        var proposedRect = CGRect(origin: .zero, size: size)

        return cgImage(forProposedRect: &proposedRect,
                       context: nil,
                       hints: nil)
    }

    convenience init?(named name: String) {
        self.init(named: Name(name))
    }
}

// Step 3: Profit - you can now make your model code that uses UIImage cross-platform!
struct User {
    let name: String
    let profileImage: UIImage
}

39 Internally mutable protocol-oriented APIs

🤖 You can easily define a protocol-oriented API that can only be mutated internally, by using an internal protocol that extends a public one.

// Declare a public protocol that acts as your immutable API
public protocol ModelHolder {
    associatedtype Model
    var model: Model { get }
}

// Declare an extended, internal protocol that provides a mutable API
internal protocol MutableModelHolder: ModelHolder {
    var model: Model { get set }
}

// You can now implement the requirements using 'public internal(set)'
public class UserHolder: MutableModelHolder {
    public internal(set) var model: User

    internal init(model: User) {
        self.model = model
    }
}

38 Switching on a set

🎛 You can switch on a set using array literals as cases in Swift! Can be really useful to avoid many if/else if statements.

class RoadTile: Tile {
    var connectedDirections = Set<Direction>()

    func render() {
        switch connectedDirections {
        case [.up, .down]:
            image = UIImage(named: "road-vertical")
        case [.left, .right]:
            image = UIImage(named: "road-horizontal")
        default:
            image = UIImage(named: "road")
        }
    }
}

37 Adding the current locale to cache keys

🌍 When caching localized content in an app, it's a good idea to add the current locale to all keys, to prevent bugs when switching languages.

func cache(_ content: Content, forKey key: String) throws {
    let data = try wrap(content) as Data
    let key = localize(key: key)
    try storage.store(data, forKey: key)
}

func loadCachedContent(forKey key: String) -> Content? {
    let key = localize(key: key)
    let data = storage.loadData(forKey: key)
    return data.flatMap { try? unbox(data: $0) }
}

private func localize(key: String) -> String {
    return key + "-" + Bundle.main.preferredLocalizations[0]
}

36 Setting up tests to avoid retain cycles with weak references

🚳 Here's an easy way to setup a test to avoid accidental retain cycles with object relationships (like weak delegates & observers) in Swift:

func testDelegateNotRetained() {
    // Assign the delegate (weak) and also retain it using a local var
    var delegate: Delegate? = DelegateMock()
    controller.delegate = delegate
    XCTAssertNotNil(controller.delegate)
    
    // Release the local var, which should also release the weak reference
    delegate = nil
    XCTAssertNil(controller.delegate)
}

35 Expressively matching a value against a list of candidates

👨‍🔬 Playing around with an expressive way to check if a value matches any of a list of candidates in Swift:

// Instead of multiple conditions like this:

if string == "One" || string == "Two" || string == "Three" {

}

// You can now do:

if string == any(of: "One", "Two", "Three") {

}

You can find a gist with the implementation here.

34 Organizing code using extensions

👪 APIs in a Swift extension automatically inherit its access control level, making it a neat way to organize public, internal & private APIs.

public extension Animation {
    init(textureNamed textureName: String) {
        frames = [Texture(name: textureName)]
    }
    
    init(texturesNamed textureNames: [String], frameDuration: TimeInterval = 1) {
        frames = textureNames.map(Texture.init)
        self.frameDuration = frameDuration
    }
    
    init(image: Image) {
        frames = [Texture(image: image)]
    }
}

internal extension Animation {
    func loadFrameImages() -> [Image] {
        return frames.map { $0.loadImageIfNeeded() }
    }
}

33 Using map to transform an optional into a Result type

🗺 Using map you can transform an optional value into an optional Result type by simply passing in the enum case.

enum Result<Value> {
    case value(Value)
    case error(Error)
}

class Promise<Value> {
    private var result: Result<Value>?
    
    init(value: Value? = nil) {
        result = value.map(Result.value)
    }
}

32 Assigning to self in struct initializers

👌 It's so nice that you can assign directly to self in struct initializers in Swift. Very useful when adding conformance to protocols.

extension Bool: AnswerConvertible {
    public init(input: String) throws {
        switch input.lowercased() {
        case "y", "yes", "👍":
            self = true
        default:
            self = false
        }
    }
}

31 Recursively calling closures as inline functions

☎️ Defining Swift closures as inline functions enables you to recursively call them, which is super useful in things like custom sequences.

class Database {
    func records(matching query: Query) -> AnySequence<Record> {
        var recordIterator = loadRecords().makeIterator()
        
        func iterate() -> Record? {
            guard let nextRecord = recordIterator.next() else {
                return nil
            }
            
            guard nextRecord.matches(query) else {
                // Since the closure is an inline function, it can be recursively called,
                // in this case in order to advance to the next item.
                return iterate()
            }
            
            return nextRecord
        }
        
        // AnySequence/AnyIterator are part of the standard library and provide an easy way
        // to define custom sequences using closures.
        return AnySequence { AnyIterator(iterate) }
    }
}

Rob Napier points out that using the above might cause crashes if used on a large databaset, since Swift has no guaranteed Tail Call Optimization (TCO).

Slava Pestov also points out that another benefit of inline functions vs closures is that they can have their own generic parameter list.

30 Passing self to required Objective-C dependencies

🏖 Using lazy properties in Swift, you can pass self to required Objective-C dependencies without having to use force-unwrapped optionals.

class DataLoader: NSObject {
    lazy var urlSession: URLSession = self.makeURLSession()
    
    private func makeURLSession() -> URLSession {
        return URLSession(configuration: .default, delegate: self, delegateQueue: .main)
    }
}

class Renderer {
    lazy var displayLink: CADisplayLink = self.makeDisplayLink()
    
    private func makeDisplayLink() -> CADisplayLink {
        return CADisplayLink(target: self, selector: #selector(screenDidRefresh))
    }
}

29 Making weak or lazy properties readonly

👓 If you have a property in Swift that needs to be weak or lazy, you can still make it readonly by using private(set).

class Node {
    private(set) weak var parent: Node?
    private(set) lazy var children = [Node]()

    func add(child: Node) {
        children.append(child)
        child.parent = self
    }
}

28 Defining static URLs using string literals

🌏 Tired of using URL(string: "url")! for static URLs? Make URL conform to ExpressibleByStringLiteral and you can now simply use "url" instead.

extension URL: ExpressibleByStringLiteral {
    // By using 'StaticString' we disable string interpolation, for safety
    public init(stringLiteral value: StaticString) {
        self = URL(string: "\(value)").require(hint: "Invalid URL string literal: \(value)")
    }
}

// We can now define URLs using static string literals 🎉
let url: URL = "https://www.swiftbysundell.com"
let task = URLSession.shared.dataTask(with: "https://www.swiftbysundell.com")

// In Swift 3 or earlier, you also have to implement 2 additional initializers
extension URL {
    public init(extendedGraphemeClusterLiteral value: StaticString) {
        self.init(stringLiteral: value)
    }

    public init(unicodeScalarLiteral value: StaticString) {
        self.init(stringLiteral: value)
    }
}

To find the extension that adds the require() method on Optional that I use above, check out Require.

27 Manipulating points, sizes and frames using math operators

✚ I'm always careful with operator overloading, but for manipulating things like sizes, points & frames I find them super useful.

extension CGSize {
    static func *(lhs: CGSize, rhs: CGFloat) -> CGSize {
        return CGSize(width: lhs.width * rhs, height: lhs.height * rhs)
    }
}

button.frame.size = image.size * 2

If you like the above idea, check out CGOperators, which contains math operator overloads for all Core Graphics' vector types.

26 Using closure types in generic constraints

🔗 You can use closure types in generic constraints in Swift. Enables nice APIs for handling sequences of closures.

extension Sequence where Element == () -> Void {
    func callAll() {
        forEach { $0() }
    }
}

extension Sequence where Element == () -> String {
    func joinedResults(separator: String) -> String {
        return map { $0() }.joined(separator: separator)
    }
}

callbacks.callAll()
let names = nameProviders.joinedResults(separator: ", ")

(If you're using Swift 3, you have to change Element to Iterator.Element)

25 Using associated enum values to avoid state-specific optionals

🎉 Using associated enum values is a super nice way to encapsulate mutually exclusive state info (and avoiding state-specific optionals).

// BEFORE: Lots of state-specific, optional properties

class Player {
    var isWaitingForMatchMaking: Bool
    var invitingUser: User?
    var numberOfLives: Int
    var playerDefeatedBy: Player?
    var roundDefeatedIn: Int?
}

// AFTER: All state-specific information is encapsulated in enum cases

class Player {
    enum State {
        case waitingForMatchMaking
        case waitingForInviteResponse(from: User)
        case active(numberOfLives: Int)
        case defeated(by: Player, roundNumber: Int)
    }
    
    var state: State
}

24 Using enums for async result types

👍 I really like using enums for all async result types, even boolean ones. Self-documenting, and makes the call site a lot nicer to read too!

protocol PushNotificationService {
    // Before
    func enablePushNotifications(completionHandler: @escaping (Bool) -> Void)
    
    // After
    func enablePushNotifications(completionHandler: @escaping (PushNotificationStatus) -> Void)
}

enum PushNotificationStatus {
    case enabled
    case disabled
}

service.enablePushNotifications { status in
    if status == .enabled {
        enableNotificationsButton.removeFromSuperview()
    }
}

23 Working on async code in a playground

🏃 Want to work on your async code in a Swift Playground? Just set needsIndefiniteExecution to true to keep it running:

import PlaygroundSupport

PlaygroundPage.current.needsIndefiniteExecution = true

DispatchQueue.main.asyncAfter(deadline: .now() + 3) {
    let greeting = "Hello after 3 seconds"
    print(greeting)
}

To stop the playground from executing, simply call PlaygroundPage.current.finishExecution().

22 Overriding self with a weak reference

💦 Avoid memory leaks when accidentially refering to self in closures by overriding it locally with a weak reference:

Swift >= 4.2

dataLoader.loadData(from: url) { [weak self] result in
    guard let self = self else { 
        return 
    }

    self.cache(result)
    
    ...

Swift < 4.2

dataLoader.loadData(from: url) { [weak self] result in
    guard let `self` = self else {
        return
    }

    self.cache(result)
    
    ...

Note that the reason the above currently works is because of a compiler bug (which I hope gets turned into a properly supported feature soon).

21 Using DispatchWorkItem

🕓 Using dispatch work items you can easily cancel a delayed asynchronous GCD task if you no longer need it:

let workItem = DispatchWorkItem {
    // Your async code goes in here
}

// Execute the work item after 1 second
DispatchQueue.main.asyncAfter(deadline: .now() + 1, execute: workItem)

// You can cancel the work item if you no longer need it
workItem.cancel()

20 Combining a sequence of functions

➕ While working on a new Swift developer tool (to be open sourced soon 😉), I came up with a pretty neat way of organizing its sequence of operations, by combining their functions into a closure:

internal func +<A, B, C>(lhs: @escaping (A) throws -> B,
                         rhs: @escaping (B) throws -> C) -> (A) throws -> C {
    return { try rhs(lhs($0)) }
}

public func run() throws {
    try (determineTarget + build + analyze + output)()
}

If you're familiar with the functional programming world, you might know the above technique as the pipe operator (thanks to Alexey Demedreckiy for pointing this out!)

19 Chaining optionals with map() and flatMap()

🗺 Using map() and flatMap() on optionals you can chain multiple operations without having to use lengthy if lets or guards:

// BEFORE

guard let string = argument(at: 1) else {
    return
}

guard let url = URL(string: string) else {
    return
}

handle(url)

// AFTER

argument(at: 1).flatMap(URL.init).map(handle)

18 Using self-executing closures for lazy properties

🚀 Using self-executing closures is a great way to encapsulate lazy property initialization:

class StoreViewController: UIViewController {
    private lazy var collectionView: UICollectionView = {
        let layout = UICollectionViewFlowLayout()
        let view = UICollectionView(frame: self.view.bounds, collectionViewLayout: layout)
        view.delegate = self
        view.dataSource = self
        return view
    }()
    
    override func viewDidLoad() {
        super.viewDidLoad()
        view.addSubview(collectionView)
    }
}

17 Speeding up Swift package tests

⚡️ You can speed up your Swift package tests using the --parallel flag. For Marathon, the tests execute 3 times faster that way!

swift test --parallel

16 Avoiding mocking UserDefaults

🛠 Struggling with mocking UserDefaults in a test? The good news is: you don't need mocking - just create a real instance:

class LoginTests: XCTestCase {
    private var userDefaults: UserDefaults!
    private var manager: LoginManager!
    
    override func setUp() {
        super.setup()
        
        userDefaults = UserDefaults(suiteName: #file)
        userDefaults.removePersistentDomain(forName: #file)
        
        manager = LoginManager(userDefaults: userDefaults)
    }
}

15 Using variadic parameters

👍 Using variadic parameters in Swift, you can create some really nice APIs that take a list of objects without having to use an array:

extension Canvas {
    func add(_ shapes: Shape...) {
        shapes.forEach(add)
    }
}

let circle = Circle(center: CGPoint(x: 5, y: 5), radius: 5)
let lineA = Line(start: .zero, end: CGPoint(x: 10, y: 10))
let lineB = Line(start: CGPoint(x: 0, y: 10), end: CGPoint(x: 10, y: 0))

let canvas = Canvas()
canvas.add(circle, lineA, lineB)
canvas.render()

14 Referring to enum cases with associated values as closures

😮 Just like you can refer to a Swift function as a closure, you can do the same thing with enum cases with associated values:

enum UnboxPath {
    case key(String)
    case keyPath(String)
}

struct UserSchema {
    static let name = key("name")
    static let age = key("age")
    static let posts = key("posts")
    
    private static let key = UnboxPath.key
}

13 Using the === operator to compare objects by instance

📈 The === operator lets you check if two objects are the same instance. Very useful when verifying that an array contains an instance in a test:

protocol InstanceEquatable: class, Equatable {}

extension InstanceEquatable {
    static func ==(lhs: Self, rhs: Self) -> Bool {
        return lhs === rhs
    }
}

extension Enemy: InstanceEquatable {}

func testDestroyingEnemy() {
    player.attack(enemy)
    XCTAssertTrue(player.destroyedEnemies.contains(enemy))
}

12 Calling initializers with dot syntax and passing them as closures

😎 Cool thing about Swift initializers: you can call them using dot syntax and pass them as closures! Perfect for mocking dates in tests.

class Logger {
    private let storage: LogStorage
    private let dateProvider: () -> Date
    
    init(storage: LogStorage = .init(), dateProvider: @escaping () -> Date = Date.init) {
        self.storage = storage
        self.dateProvider = dateProvider
    }
    
    func log(event: Event) {
        storage.store(event: event, date: dateProvider())
    }
}

11 Structuring UI tests as extensions on XCUIApplication

📱 Most of my UI testing logic is now categories on XCUIApplication. Makes the test cases really easy to read:

func testLoggingInAndOut() {
    XCTAssertFalse(app.userIsLoggedIn)
    
    app.launch()
    app.login()
    XCTAssertTrue(app.userIsLoggedIn)
    
    app.logout()
    XCTAssertFalse(app.userIsLoggedIn)
}

func testDisplayingCategories() {
    XCTAssertFalse(app.isDisplayingCategories)
    
    app.launch()
    app.login()
    app.goToCategories()
    XCTAssertTrue(app.isDisplayingCategories)
}

10 Avoiding default cases in switch statements

🙂 It’s a good idea to avoid “default” cases when switching on Swift enums - it’ll “force you” to update your logic when a new case is added:

enum State {
    case loggedIn
    case loggedOut
    case onboarding
}

func handle(_ state: State) {
    switch state {
    case .loggedIn:
        showMainUI()
    case .loggedOut:
        showLoginUI()
    // Compiler error: Switch must be exhaustive
    }
}

9 Using the guard statement in many different scopes

💂 It's really cool that you can use Swift's 'guard' statement to exit out of pretty much any scope, not only return from functions:

// You can use the 'guard' statement to...

for string in strings {
    // ...continue an iteration
    guard shouldProcess(string) else {
        continue
    }
    
    // ...or break it
    guard !shouldBreak(for: string) else {
        break
    }
    
    // ...or return
    guard !shouldReturn(for: string) else {
        return
    }
    
    // ..or throw an error
    guard string.isValid else {
        throw StringError.invalid(string)
    }
    
    // ...or exit the program
    guard !shouldExit(for: string) else {
        exit(1)
    }
}

8 Passing functions & operators as closures

❤️ Love how you can pass functions & operators as closures in Swift. For example, it makes the syntax for sorting arrays really nice!

let array = [3, 9, 1, 4, 6, 2]
let sorted = array.sorted(by: <)

7 Using #function for UserDefaults key consistency

🗝 Here's a neat little trick I use to get UserDefault key consistency in Swift (#function expands to the property name in getters/setters). Just remember to write a good suite of tests that'll guard you against bugs when changing property names.

extension UserDefaults {
    var onboardingCompleted: Bool {
        get { return bool(forKey: #function) }
        set { set(newValue, forKey: #function) }
    }
}

6 Using a name already taken by the standard library

📛 Want to use a name already taken by the standard library for a nested type? No problem - just use Swift. to disambiguate:

extension Command {
    enum Error: Swift.Error {
        case missing
        case invalid(String)
    }
}

5 Using Wrap to implement Equatable

📦 Playing around with using Wrap to implement Equatable for any type, primarily for testing:

protocol AutoEquatable: Equatable {}

extension AutoEquatable {
    static func ==(lhs: Self, rhs: Self) -> Bool {
        let lhsData = try! wrap(lhs) as Data
        let rhsData = try! wrap(rhs) as Data
        return lhsData == rhsData
    }
}

4 Using typealiases to reduce the length of method signatures

📏 One thing that I find really useful in Swift is to use typealiases to reduce the length of method signatures in generic types:

public class PathFinder<Object: PathFinderObject> {
    public typealias Map = Object.Map
    public typealias Node = Map.Node
    public typealias Path = PathFinderPath<Object>
    
    public static func possiblePaths(for object: Object, at rootNode: Node, on map: Map) -> Path.Sequence {
        return .init(object: object, rootNode: rootNode, map: map)
    }
}

3 Referencing either external or internal parameter name when writing docs

📖 You can reference either the external or internal parameter label when writing Swift docs - and they get parsed the same:

// EITHER:

class Foo {
    /**
    *   - parameter string: A string
    */
    func bar(with string: String) {}
}

// OR:

class Foo {
    /**
    *   - parameter with: A string
    */
    func bar(with string: String) {}
}

2 Using auto closures

👍 Finding more and more uses for auto closures in Swift. Can enable some pretty nice APIs:

extension Dictionary {
    mutating func value(for key: Key, orAdd valueClosure: @autoclosure () -> Value) -> Value {
        if let value = self[key] {
            return value
        }
        
        let value = valueClosure()
        self[key] = value
        return value
    }
}

1 Namespacing with nested types

🚀 I’ve started to become a really big fan of nested types in Swift. Love the additional namespacing it gives you!

public struct Map {
    public struct Model {
        public let size: Size
        public let theme: Theme
        public var terrain: [Position : Terrain.Model]
        public var units: [Position : Unit.Model]
        public var buildings: [Position : Building.Model]
    }
    
    public enum Direction {
        case up
        case right
        case down
        case left
    }
    
    public struct Position {
        public var x: Int
        public var y: Int
    }
    
    public enum Size: String {
        case small = "S"
        case medium = "M"
        case large = "L"
        case extraLarge = "XL"
    }
}

Download Details:

Author: JohnSundell
Source Code: https://github.com/JohnSundell/SwiftTips 
License: MIT license

#swift #tips #tricks 

A Collection Of Swift Tips & Tricks That I've Shared on Twitter

⚠️ This list is no longer being updated. For my latest Swift tips, checkout the "Tips" section on Swift by Sundell.

Swift tips & tricks ⚡️

One of the things I really love about Swift is how I keep finding interesting ways to use it in various situations, and when I do - I usually share them on Twitter. Here's a collection of all the tips & tricks that I've shared so far. Each entry has a link to the original tweet, if you want to respond with some feedback or question, which is always super welcome! 🚀

Also make sure to check out all of my other Swift content:

• 📖 My weekly articles, with a new post about Swift every Sunday!
• 🎧 My podcast which features special guests from around the community.
• 📬 My monthly newsletter - rounding up all of my content, including these tips, on the 1st of every month.

#102 Making async tests faster and more stable

🚀 Here are some quick tips to make async tests faster & more stable:

• 😴 Avoid sleep() - use expectations instead
• ⏱ Use generous timeouts to avoid flakiness on CI
• 🧐 Put all assertions at the end of each test, not inside closures

// BEFORE:

class MentionDetectorTests: XCTestCase {
    func testDetectingMention() {
        let detector = MentionDetector()
        let string = "This test was written by @johnsundell."

        detector.detectMentions(in: string) { mentions in
            XCTAssertEqual(mentions, ["johnsundell"])
        }
        
        sleep(2)
    }
}

// AFTER:

class MentionDetectorTests: XCTestCase {
    func testDetectingMention() {
        let detector = MentionDetector()
        let string = "This test was written by @johnsundell."

        var mentions: [String]?
        let expectation = self.expectation(description: #function)

        detector.detectMentions(in: string) {
            mentions = $0
            expectation.fulfill()
        }

        waitForExpectations(timeout: 10)
        XCTAssertEqual(mentions, ["johnsundell"])
    }
}

For more on async testing, check out "Unit testing asynchronous Swift code".

#101 Adding support for Apple Pencil double-taps

✍️ Adding support for the new Apple Pencil double-tap feature is super easy! All you have to do is to create a UIPencilInteraction, add it to a view, and implement one delegate method. Hopefully all pencil-compatible apps will soon adopt this.

let interaction = UIPencilInteraction()
interaction.delegate = self
view.addInteraction(interaction)

extension ViewController: UIPencilInteractionDelegate {
    func pencilInteractionDidTap(_ interaction: UIPencilInteraction) {
        // Handle pencil double-tap
    }
}

For more on using this and other iPad Pro features, check out "Building iPad Pro features in Swift".

#100 Combining values with functions

😎 Here's a cool function that combines a value with a function to return a closure that captures that value, so that it can be called without any arguments. Super useful when working with closure-based APIs and we want to use some of our properties without having to capture self.

func combine<A, B>(_ value: A, with closure: @escaping (A) -> B) -> () -> B {
    return { closure(value) }
}

// BEFORE:

class ProductViewController: UIViewController {
    override func viewDidLoad() {
        super.viewDidLoad()

        buyButton.handler = { [weak self] in
            guard let self = self else {
                return
            }
            
            self.productManager.startCheckout(for: self.product)
        }
    }
}

// AFTER:

class ProductViewController: UIViewController {
    override func viewDidLoad() {
        super.viewDidLoad()

        buyButton.handler = combine(product, with: productManager.startCheckout)
    }
}

#99 Dependency injection using functions

💉 When I'm only using a single function from a dependency, I love to inject that function as a closure, instead of having to create a protocol and inject the whole object. Makes dependency injection & testing super simple.

final class ArticleLoader {
    typealias Networking = (Endpoint) -> Future<Data>
    
    private let networking: Networking
    
    init(networking: @escaping Networking = URLSession.shared.load) {
        self.networking = networking
    }
    
    func loadLatest() -> Future<[Article]> {
        return networking(.latestArticles).decode()
    }
}

For more on this technique, check out "Simple Swift dependency injection with functions".

#98 Using a custom exception handler

💥 It's cool that you can easily assign a closure as a custom NSException handler. This is super useful when building things in Playgrounds - since you can't use breakpoints - so instead of just signal SIGABRT, you'll get the full exception description if something goes wrong.

NSSetUncaughtExceptionHandler { exception in
    print(exception)
}

#97 Using type aliases to give semantic meaning to primitives

❤️ I love that in Swift, we can use the type system to make our code so much more self-documenting - one way of doing so is to use type aliases to give the primitive types that we use a more semantic meaning.

extension List.Item {
    // Using type aliases, we can give semantic meaning to the
    // primitive types that we use, without having to introduce
    // wrapper types.
    typealias Index = Int
}

extension List {
    enum Mutation {
        // Our enum cases now become a lot more self-documenting,
        // without having to add additional parameter labels to
        // explain them.
        case add(Item, Item.Index)
        case update(Item, Item.Index)
        case remove(Item.Index)
    }
}

For more on self-documenting code, check out "Writing self-documenting Swift code".

#96 Specializing protocols using constraints

🤯 A little late night prototyping session reveals that protocol constraints can not only be applied to extensions - they can also be added to protocol definitions!

This is awesome, since it lets us easily define specialized protocols based on more generic ones.

protocol Component {
    associatedtype Container
    func add(to container: Container)
}

// Protocols that inherit from other protocols can include
// constraints to further specialize them.
protocol ViewComponent: Component where Container == UIView {
    associatedtype View: UIView
    var view: View { get }
}

extension ViewComponent {
    func add(to container: UIView) {
        container.addSubview(view)
    }
}

For more on specializing protocols, check out "Specializing protocols in Swift".

#95 Unwrapping an optional or throwing an error

📦 Here's a super handy extension on Swift's Optional type, which gives us a really nice API for easily unwrapping an optional, or throwing an error in case the value turned out to be nil:

extension Optional {
    func orThrow(_ errorExpression: @autoclosure () -> Error) throws -> Wrapped {
        switch self {
        case .some(let value):
            return value
        case .none:
            throw errorExpression()
        }
    }
}

let file = try loadFile(at: path).orThrow(MissingFileError())

For more ways that optionals can be extended, check out "Extending optionals in Swift".

#94 Testing code that uses static APIs

👩‍🔬 Testing code that uses static APIs can be really tricky, but there's a way that it can often be done - using Swift's first class function capabilities!

Instead of accessing that static API directly, we can inject the function we want to use, which enables us to mock it!

// BEFORE

class FriendsLoader {
    func loadFriends(then handler: @escaping (Result<[Friend]>) -> Void) {
        Networking.loadData(from: .friends) { result in
            ...
        }
    }
}

// AFTER

class FriendsLoader {
    typealias Handler<T> = (Result<T>) -> Void
    typealias DataLoadingFunction = (Endpoint, @escaping Handler<Data>) -> Void

    func loadFriends(using dataLoading: DataLoadingFunction = Networking.loadData,
                     then handler: @escaping Handler<[Friend]>) {
        dataLoading(.friends) { result in
            ...
        }
    }
}

// MOCKING IN TESTS

let dataLoading: FriendsLoader.DataLoadingFunction = { _, handler in
    handler(.success(mockData))
}

friendsLoader.loadFriends(using: dataLoading) { result in
    ...
}

#93 Matching multiple enum cases with associated values

🐾 Swift's pattern matching capabilities are so powerful! Two enum cases with associated values can even be matched and handled by the same switch case - which is super useful when handling state changes with similar data.

enum DownloadState {
    case inProgress(progress: Double)
    case paused(progress: Double)
    case cancelled
    case finished(Data)
}

func downloadStateDidChange(to state: DownloadState) {
    switch state {
    case .inProgress(let progress), .paused(let progress):
        updateProgressView(with: progress)
    case .cancelled:
        showCancelledMessage()
    case .finished(let data):
        process(data)
    }
}

#92 Multiline string literals

🅰 One really nice benefit of Swift multiline string literals - even for single lines of text - is that they don't require quotes to be escaped. Perfect when working with things like HTML, or creating a custom description for an object.

let html = highlighter.highlight("Array<String>")

XCTAssertEqual(html, """
<span class="type">Array</span>&lt;<span class="type">String</span>&gt;
""")

#91 Reducing sequences

💎 While it's very common in functional programming, the reduce function might be a bit of a hidden gem in Swift. It provides a super useful way to transform a sequence into a single value.

extension Sequence where Element: Equatable {
    func numberOfOccurrences(of target: Element) -> Int {
        return reduce(0) { result, element in
            guard element == target else {
                return result
            }

            return result + 1
        }
    }
}

You can read more about transforming collections in "Transforming collections in Swift".

#90 Avoiding manual Codable implementations

📦 When I use Codable in Swift, I want to avoid manual implementations as much as possible, even when there's a mismatch between my code structure and the JSON I'm decoding.

One way that can often be achieved is to use private data containers combined with computed properties.

struct User: Codable {
    let name: String
    let age: Int

    var homeTown: String { return originPlace.name }

    private let originPlace: Place
}

private extension User {
    struct Place: Codable {
        let name: String
    }
}

extension User {
    struct Container: Codable {
        let user: User
    }
}

#89 Using feature flags instead of feature branches

🚢 Instead of using feature branches, I merge almost all of my code directly into master - and then I use feature flags to conditionally enable features when they're ready. That way I can avoid merge conflicts and keep shipping!

extension ListViewController {
    func addSearchIfNeeded() {
        // Rather than having to keep maintaining a separate
        // feature branch for a new feature, we can use a flag
        // to conditionally turn it on.
        guard FeatureFlags.searchEnabled else {
            return
        }

        let resultsVC = SearchResultsViewController()
        let searchVC = UISearchController(
            searchResultsController: resultsVC
        )

        searchVC.searchResultsUpdater = resultsVC
        navigationItem.searchController = searchVC
    }
}

You can read more about feature flags in "Feature flags in Swift".

#88 Lightweight data hierarchies using tuples

💾 Here I'm using tuples to create a lightweight hierarchy for my data, giving me a nice structure without having to introduce any additional types.

struct CodeSegment {
    var tokens: (
        previous: String?,
        current: String
    )

    var delimiters: (
        previous: Character?
        next: Character?
    )
}

handle(segment.tokens.current)

You can read more about tuples in "Using tuples as lightweight types in Swift"

#87 The rule of threes

3️⃣ Whenever I have 3 properties or local variables that share the same prefix, I usually try to extract them into their own method or type. That way I can avoid massive types & methods, and also increase readability, without falling into a "premature optimization" trap.

Before

public func generate() throws {
    let contentFolder = try folder.subfolder(named: "content")

    let articleFolder = try contentFolder.subfolder(named: "posts")
    let articleProcessor = ContentProcessor(folder: articleFolder)
    let articles = try articleProcessor.process()

    ...
}

After

public func generate() throws {
    let contentFolder = try folder.subfolder(named: "content")
    let articles = try processArticles(in: contentFolder)
    ...
}

private func processArticles(in folder: Folder) throws -> [ContentItem] {
    let folder = try folder.subfolder(named: "posts")
    let processor = ContentProcessor(folder: folder)
    return try processor.process()
}

#86 Useful Codable extensions

👨‍🔧 Here's two extensions that I always add to the Encodable & Decodable protocols, which for me really make the Codable API nicer to use. By using type inference for decoding, a lot of boilerplate can be removed when the compiler is already able to infer the resulting type.

extension Encodable {
    func encoded() throws -> Data {
        return try JSONEncoder().encode(self)
    }
}

extension Data {
    func decoded<T: Decodable>() throws -> T {
        return try JSONDecoder().decode(T.self, from: self)
    }
}

let data = try user.encoded()

// By using a generic type in the decoded() method, the
// compiler can often infer the type we want to decode
// from the current context.
try userDidLogin(data.decoded())

// And if not, we can always supply the type, still making
// the call site read very nicely.
let otherUser = try data.decoded() as User

#85 Using shared UserDefaults suites

📦 UserDefaults is a lot more powerful than what it first might seem like. Not only can it store more complex values (like dates & dictionaries) and parse command line arguments - it also enables easy sharing of settings & lightweight data between apps in the same App Group.

let sharedDefaults = UserDefaults(suiteName: "my-app-group")!
let useDarkMode = sharedDefaults.bool(forKey: "dark-mode")

// This value is put into the shared suite.
sharedDefaults.set(true, forKey: "dark-mode")

// If you want to treat the shared settings as read-only (and add
// local overrides on top of them), you can simply add the shared
// suite to the standard UserDefaults.
let combinedDefaults = UserDefaults.standard
combinedDefaults.addSuite(named: "my-app-group")

// This value is a local override, not added to the shared suite.
combinedDefaults.set(true, forKey: "app-specific-override")

#84 Custom UIView backing layers

🎨 By overriding layerClass you can tell UIKit what CALayer class to use for a UIView's backing layer. That way you can reduce the amount of layers, and don't have to do any manual layout.

final class GradientView: UIView {
    override class var layerClass: AnyClass { return CAGradientLayer.self }

    var colors: (start: UIColor, end: UIColor)? {
        didSet { updateLayer() }
    }

    private func updateLayer() {
        let layer = self.layer as! CAGradientLayer
        layer.colors = colors.map { [$0.start.cgColor, $0.end.cgColor] }
    }
}

#83 Auto-Equatable enums with associated values

✅ That the compiler now automatically synthesizes Equatable conformances is such a huge upgrade for Swift! And the cool thing is that it works for all kinds of types - even for enums with associated values! Especially useful when using enums for verification in unit tests.

struct Article: Equatable {
    let title: String
    let text: String
}

struct User: Equatable {
    let name: String
    let age: Int
}

extension Navigator {
    enum Destination: Equatable {
        case profile(User)
        case article(Article)
    }
}

func testNavigatingToArticle() {
    let article = Article(title: "Title", text: "Text")
    controller.select(article)
    XCTAssertEqual(navigator.destinations, [.article(article)])
}

#82 Defaults for associated types

🤝 Associated types can have defaults in Swift - which is super useful for types that are not easily inferred (for example when they're not used for a specific instance method or property).

protocol Identifiable {
    associatedtype RawIdentifier: Codable = String

    var id: Identifier<Self> { get }
}

struct User: Identifiable {
    let id: Identifier<User>
    let name: String
}

struct Group: Identifiable {
    typealias RawIdentifier = Int

    let id: Identifier<Group>
    let name: String
}

#81 Creating a dedicated identifier type

🆔 If you want to avoid using plain strings as identifiers (which can increase both type safety & readability), it's really easy to create a custom Identifier type that feels just like a native Swift type, thanks to protocols!

More on this topic in "Type-safe identifiers in Swift".

struct Identifier: Hashable {
    let string: String
}

extension Identifier: ExpressibleByStringLiteral {
    init(stringLiteral value: String) {
        string = value
    }
}

extension Identifier: CustomStringConvertible {
    var description: String {
        return string
    }
}

extension Identifier: Codable {
    init(from decoder: Decoder) throws {
        let container = try decoder.singleValueContainer()
        string = try container.decode(String.self)
    }

    func encode(to encoder: Encoder) throws {
        var container = encoder.singleValueContainer()
        try container.encode(string)
    }
}

struct Article: Codable {
    let id: Identifier
    let title: String
}

let article = Article(id: "my-article", title: "Hello world!")

#80 Assigning optional tuple members to variables

🙌 A really cool thing about using tuples to model the internal state of a Swift type, is that you can unwrap an optional tuple's members directly into local variables.

Very useful in order to group multiple optional values together for easy unwrapping & handling.

class ImageTransformer {
    private var queue = [(image: UIImage, transform: Transform)]()

    private func processNext() {
        // When unwrapping an optional tuple, you can assign the members
        // directly to local variables.
        guard let (image, transform) = queue.first else {
            return
        }

        let context = Context()
        context.draw(image)
        context.apply(transform)
        ...
    }
}

#79 Struct convenience initializers

❤️ I love to structure my code using extensions in Swift. One big benefit of doing so when it comes to struct initializers, is that defining a convenience initializer doesn't remove the default one the compiler generates - best of both worlds!

struct Article {
    let date: Date
    var title: String
    var text: String
    var comments: [Comment]
}

extension Article {
    init(title: String, text: String) {
        self.init(date: Date(), title: title, text: text, comments: [])
    }
}

let articleA = Article(title: "Best Cupcake Recipe", text: "...")

let articleB = Article(
    date: Date(),
    title: "Best Cupcake Recipe",
    text: "...",
    comments: [
        Comment(user: currentUser, text: "Yep, can confirm!")
    ]
)

#78 Usages of throwing functions

🏈 A big benefit of using throwing functions for synchronous Swift APIs is that the caller can decide whether they want to treat the return value as optional (try?) or required (try).

func loadFile(named name: String) throws -> File {
    guard let url = urlForFile(named: name) else {
        throw File.Error.missing
    }

    do {
        let data = try Data(contentsOf: url)
        return File(url: url, data: data)
    } catch {
        throw File.Error.invalidData(error)
    }
}

let requiredFile = try loadFile(named: "AppConfig.json")

let optionalFile = try? loadFile(named: "UserSettings.json")

#77 Nested generic types

🐝 Types that are nested in generics automatically inherit their parent's generic types - which is super useful when defining accessory types (for things like states or outcomes).

struct Task<Input, Output> {
    typealias Closure = (Input) throws -> Output

    let closure: Closure
}

extension Task {
    enum Result {
        case success(Output)
        case failure(Error)
    }
}

#76 Equatable & Hashable structures

🤖 Now that the Swift compiler automatically synthesizes Equatable & Hashable conformances for value types, it's easier than ever to setup model structures with nested types that are all Equatable/Hashable!

typealias Value = Hashable & Codable

struct User: Value {
    var name: String
    var age: Int
    var lastLoginDate: Date?
    var settings: Settings
}

extension User {
    struct Settings: Value {
        var itemsPerPage: Int
        var theme: Theme
    }
}

extension User.Settings {
    enum Theme: String, Value {
        case light
        case dark
    }
}

You can read more about using nested types in Swift here.

#75 Conditional conformances

🎉 Swift 4.1 is here! One of the key features it brings is conditional conformances, which lets you have a type only conform to a protocol under certain constraints.

protocol UnboxTransformable {
    associatedtype RawValue

    static func transform(_ value: RawValue) throws -> Self?
}

extension Array: UnboxTransformable where Element: UnboxTransformable {
    typealias RawValue = [Element.RawValue]

    static func transform(_ value: RawValue) throws -> [Element]? {
        return try value.compactMap(Element.transform)
    }
}

I also have an article with lots of more info on conditional conformances here. Paul Hudson also has a great overview of all Swift 4.1 features here.

#74 Generic type aliases

🕵️‍♀️ A cool thing about Swift type aliases is that they can be generic! Combine that with tuples and you can easily define simple generic types.

typealias Pair<T> = (T, T)

extension Game {
    func calculateScore(for players: Pair<Player>) -> Int {
        ...
    }
}

You can read more about using tuples as lightweight types here.

#73 Parsing command line arguments using UserDefaults

☑️ A really cool "hidden" feature of UserDefaults is that it contains any arguments that were passed to the app at launch!

Super useful both in Swift command line tools & scripts, but also to temporarily override a value when debugging iOS apps.

let defaults = UserDefaults.standard
let query = defaults.string(forKey: "query")
let resultCount = defaults.integer(forKey: "results")

#72 Using the & operator

👏 Swift's & operator is awesome! Not only can you use it to compose protocols, you can compose other types too! Very useful if you want to hide concrete types & implementation details.

protocol LoadableFromURL {
    func load(from url: URL)
}

class ContentViewController: UIViewController, LoadableFromURL {
    func load(from url: URL) {
        ...
    }
}

class ViewControllerFactory {
    func makeContentViewController() -> UIViewController & LoadableFromURL {
        return ContentViewController()
    }
}

#71 Capturing multiple values in mocks

🤗 When capturing values in mocks, using an array (instead of just a single value) makes it easy to verify that only a certain number of values were passed.

Perfect for protecting against "over-calling" something.

class UserManagerTests: XCTestCase {
    func testObserversCalledWhenUserFirstLogsIn() {
        let manager = UserManager()

        let observer = ObserverMock()
        manager.addObserver(observer)

        // First login, observers should be notified
        let user = User(id: 123, name: "John")
        manager.userDidLogin(user)
        XCTAssertEqual(observer.users, [user])

        // If the same user logs in again, observers shouldn't be notified
        manager.userDidLogin(user)
        XCTAssertEqual(observer.users, [user])
    }
}

private extension UserManagerTests {
    class ObserverMock: UserManagerObserver {
        private(set) var users = [User]()

        func userDidChange(to user: User) {
            users.append(user)
        }
    }
}

#70 Reducing the need for mocks

👋 When writing tests, you don't always need to create mocks - you can create stubs using real instances of things like errors, URLs & UserDefaults.

Here's how to do that for some common tasks/object types in Swift:

// Create errors using NSError (#function can be used to reference the name of the test)
let error = NSError(domain: #function, code: 1, userInfo: nil)

// Create non-optional URLs using file paths
let url = URL(fileURLWithPath: "Some/URL")

// Reference the test bundle using Bundle(for:)
let bundle = Bundle(for: type(of: self))

// Create an explicit UserDefaults object (instead of having to use a mock)
let userDefaults = UserDefaults(suiteName: #function)

// Create queues to control/await concurrent operations
let queue = DispatchQueue(label: #function)

For when you actually do need mocking, check out "Mocking in Swift".

#69 Using "then" as an external parameter label for closures

⏱ I've started using "then" as an external parameter label for completion handlers. Makes the call site read really nicely (Because I do ❤️ conversational API design) regardless of whether trailing closure syntax is used or not.

protocol DataLoader {
    // Adding type aliases to protocols can be a great way to
    // reduce verbosity for parameter types.
    typealias Handler = (Result<Data>) -> Void
    associatedtype Endpoint

    func loadData(from endpoint: Endpoint, then handler: @escaping Handler)
}

loader.loadData(from: .messages) { result in
    ...
}

loader.loadData(from: .messages, then: { result in
    ...
})

#68 Combining lazily evaluated sequences with the builder pattern

😴 Combining lazily evaluated sequences with builder pattern-like properties can lead to some pretty sweet APIs for configurable sequences in Swift.

Also useful for queries & other things you "build up" and then execute.

// Extension adding builder pattern-like properties that return
// a new sequence value with the given configuration applied
extension FileSequence {
    var recursive: FileSequence {
        var sequence = self
        sequence.isRecursive = true
        return sequence
    }

    var includingHidden: FileSequence {
        var sequence = self
        sequence.includeHidden = true
        return sequence
    }
}

// BEFORE

let files = folder.makeFileSequence(recursive: true, includeHidden: true)

// AFTER

let files = folder.files.recursive.includingHidden

Want an intro to lazy sequences? Check out "Swift sequences: The art of being lazy".

#67 Faster & more stable UI tests

My top 3 tips for faster & more stable UI tests:

📱 Reset the app's state at the beginning of every test.

🆔 Use accessibility identifiers instead of UI strings.

⏱ Use expectations instead of waiting time.

func testOpeningArticle() {
    // Launch the app with an argument that tells it to reset its state
    let app = XCUIApplication()
    app.launchArguments.append("--uitesting")
    app.launch()
    
    // Check that the app is displaying an activity indicator
    let activityIndicator = app.activityIndicator.element
    XCTAssertTrue(activityIndicator.exists)
    
    // Wait for the loading indicator to disappear = content is ready
    expectation(for: NSPredicate(format: "exists == 0"),
                evaluatedWith: activityIndicator)
                
    // Use a generous timeout in case the network is slow
    waitForExpectations(timeout: 10)
    
    // Tap the cell for the first article
    app.tables.cells["Article.0"].tap()
    
    // Assert that a label with the accessibility identifier "Article.Title" exists
    let label = app.staticTexts["Article.Title"]
    XCTAssertTrue(label.exists)
}

#66 Accessing the clipboard from a Swift script

📋 It's super easy to access the contents of the clipboard from a Swift script. A big benefit of Swift scripting is being able to use Cocoa's powerful APIs for Mac apps.

import Cocoa

let clipboard = NSPasteboard.general.string(forType: .string)

#65 Using tuples for view state

🎯 Using Swift tuples for view state can be a super nice way to group multiple properties together and render them reactively using the layout system.

By using a tuple we don't have to either introduce a new type or make our view model-aware.

class TextView: UIView {
    var state: (title: String?, text: String?) {
        // By telling UIKit that our view needs layout and binding our
        // state in layoutSubviews, we can react to state changes without
        // doing unnecessary layout work.
        didSet { setNeedsLayout() }
    }

    private let titleLabel = UILabel()
    private let textLabel = UILabel()

    override func layoutSubviews() {
        super.layoutSubviews()

        titleLabel.text = state.title
        textLabel.text = state.text

        ...
    }
}

#64 Throwing tests and LocalizedError

⚾️ Swift tests can throw, which is super useful in order to avoid complicated logic or force unwrapping. By making errors conform to LocalizedError, you can also get a nice error message in Xcode if there's a failure.

class ImageCacheTests: XCTestCase {
    func testCachingAndLoadingImage() throws {
        let bundle = Bundle(for: type(of: self))
        let cache = ImageCache(bundle: bundle)
        
        // Bonus tip: You can easily load images from your test
        // bundle using this UIImage initializer
        let image = try require(UIImage(named: "sample", in: bundle, compatibleWith: nil))
        try cache.cache(image, forKey: "key")
        
        let cachedImage = try cache.image(forKey: "key")
        XCTAssertEqual(image, cachedImage)
    }
}

enum ImageCacheError {
    case emptyKey
    case dataConversionFailed
}

// When using throwing tests, making your errors conform to
// LocalizedError will render a much nicer error message in
// Xcode (per default only the error code is shown).
extension ImageCacheError: LocalizedError {
    var errorDescription: String? {
        switch self {
        case .emptyKey:
            return "An empty key was given"
        case .dataConversionFailed:
            return "Failed to convert the given image to Data"
        }
    }
}

For more information, and the implementation of the require method used above, check out "Avoiding force unwrapping in Swift unit tests".

#63 The difference between static and class properties

✍️ Unlike static properties, class properties can be overridden by subclasses (however, they can't be stored, only computed).

class TableViewCell: UITableViewCell {
    class var preferredHeight: CGFloat { return 60 }
}

class TallTableViewCell: TableViewCell {
    override class var preferredHeight: CGFloat { return 100 }
}

#62 Creating extensions with static factory methods

👨‍🎨 Creating extensions with static factory methods can be a great alternative to subclassing in Swift, especially for things like setting up UIViews, CALayers or other kinds of styling.

It also lets you remove a lot of styling & setup from your view controllers.

extension UILabel {
    static func makeForTitle() -> UILabel {
        let label = UILabel()
        label.font = .boldSystemFont(ofSize: 24)
        label.textColor = .darkGray
        label.adjustsFontSizeToFitWidth = true
        label.minimumScaleFactor = 0.75
        return label
    }

    static func makeForText() -> UILabel {
        let label = UILabel()
        label.font = .systemFont(ofSize: 16)
        label.textColor = .black
        label.numberOfLines = 0
        return label
    }
}

class ArticleViewController: UIViewController {
    lazy var titleLabel = UILabel.makeForTitle()
    lazy var textLabel = UILabel.makeForText()
}

#61 Child view controller auto-resizing

🧒 An awesome thing about child view controllers is that they're automatically resized to match their parent, making them a super nice solution for things like loading & error views.

class ListViewController: UIViewController {
    func loadItems() {
        let loadingViewController = LoadingViewController()
        add(loadingViewController)

        dataLoader.loadItems { [weak self] result in
            loadingViewController.remove()
            self?.handle(result)
        }
    }
}

For more about child view controller (including the add and remove methods used above), check out "Using child view controllers as plugins in Swift".

#60 Using zip

🤐 Using the zip function in Swift you can easily combine two sequences. Super useful when using two sequences to do some work, since zip takes care of all the bounds-checking.

func render(titles: [String]) {
    for (label, text) in zip(titleLabels, titles) {
        print(text)
        label.text = text
    }
}

#59 Defining custom option sets

🎛 The awesome thing about option sets in Swift is that they can automatically either be passed as a single member or as a set. Even cooler is that you can easily define your own option sets as well, perfect for options and other non-exclusive values.

// Option sets are awesome, because you can easily pass them
// both using dot syntax and array literal syntax, like when
// using the UIView animation API:
UIView.animate(withDuration: 0.3,
               delay: 0,
               options: .allowUserInteraction,
               animations: animations)

UIView.animate(withDuration: 0.3,
               delay: 0,
               options: [.allowUserInteraction, .layoutSubviews],
               animations: animations)

// The cool thing is that you can easily define your own option
// sets as well, by defining a struct that has an Int rawValue,
// that will be used as a bit mask.
extension Cache {
    struct Options: OptionSet {
        static let saveToDisk = Options(rawValue: 1)
        static let clearOnMemoryWarning = Options(rawValue: 1 << 1)
        static let clearDaily = Options(rawValue: 1 << 2)

        let rawValue: Int
    }
}

// We can now use Cache.Options just like UIViewAnimationOptions:
Cache(options: .saveToDisk)
Cache(options: [.saveToDisk, .clearDaily])

#58 Using the where clause with associated types

🙌 Using the where clause when designing protocol-oriented APIs in Swift can let your implementations (or others' if it's open source) have a lot more freedom, especially when it comes to collections.

See "Using generic type constraints in Swift 4" for more info.

public protocol PathFinderMap {
    associatedtype Node
    // Using the 'where' clause for associated types, we can
    // ensure that a type meets certain requirements (in this
    // case that it's a sequence with Node elements).
    associatedtype NodeSequence: Sequence where NodeSequence.Element == Node

    // Instead of using a concrete type (like [Node]) here, we
    // give implementors of this protocol more freedom while
    // still meeting our requirements. For example, one
    // implementation might use Set<Node>.
    func neighbors(of node: Node) -> NodeSequence
}

#57 Using first class functions when iterating over a dictionary

👨‍🍳 Combine first class functions in Swift with the fact that Dictionary elements are (Key, Value) tuples and you can build yourself some pretty awesome functional chains when iterating over a Dictionary.

func makeActor(at coordinate: Coordinate, for building: Building) -> Actor {
    let actor = Actor()
    actor.position = coordinate.point
    actor.animation = building.animation
    return actor
}

func render(_ buildings: [Coordinate : Building]) {
    buildings.map(makeActor).forEach(add)
}

#56 Calling instance methods as static functions

😎 In Swift, you can call any instance method as a static function and it will return a closure representing that method. This is how running tests using SPM on Linux works.

More about this topic in my blog post "First class functions in Swift".

// This produces a '() -> Void' closure which is a reference to the
// given view's 'removeFromSuperview' method.
let closure = UIView.removeFromSuperview(view)

// We can now call it just like we would any other closure, and it
// will run 'view.removeFromSuperview()'
closure()

// This is how running tests using the Swift Package Manager on Linux
// works, you return your test functions as closures:
extension UserManagerTests {
    static var allTests = [
        ("testLoggingIn", testLoggingIn),
        ("testLoggingOut", testLoggingOut),
        ("testUserPermissions", testUserPermissions)
    ]
}

#55 Dropping suffixes from method names to support multiple arguments

👏 One really nice benefit of dropping suffixes from method names (and just using verbs, when possible) is that it becomes super easy to support both single and multiple arguments, and it works really well semantically.

extension UIView {
    func add(_ subviews: UIView...) {
        subviews.forEach(addSubview)
    }
}

view.add(button)
view.add(label)

// By dropping the "Subview" suffix from the method name, both
// single and multiple arguments work really well semantically.
view.add(button, label)

#54 Constraining protocols to classes to ensure mutability

👽 Using the AnyObject (or class) constraint on protocols is not only useful when defining delegates (or other weak references), but also when you always want instances to be mutable without copying.

// By constraining a protocol with 'AnyObject' it can only be adopted
// by classes, which means all instances will always be mutable, and
// that it's the original instance (not a copy) that will be mutated.
protocol DataContainer: AnyObject {
    var data: Data? { get set }
}

class UserSettingsManager {
    private var settings: Settings
    private let dataContainer: DataContainer

    // Since DataContainer is a protocol, we an easily mock it in
    // tests if we use dependency injection
    init(settings: Settings, dataContainer: DataContainer) {
        self.settings = settings
        self.dataContainer = dataContainer
    }

    func saveSettings() throws {
        let data = try settings.serialize()

        // We can now assign properties on an instance of our protocol
        // because the compiler knows it's always going to be a class
        dataContainer.data = data
    }
}

#53 String-based enums in string interpolation

🍣 Even if you define a custom raw value for a string-based enum in Swift, the full case name will be used in string interpolation.

Super useful when using separate raw values for JSON, while still wanting to use the full case name in other contexts.

extension Building {
    // This enum has custom raw values that are used when decoding
    // a value, for example from JSON.
    enum Kind: String {
        case castle = "C"
        case town = "T"
        case barracks = "B"
        case goldMine = "G"
        case camp = "CA"
        case blacksmith = "BL"
    }

    var animation: Animation {
        return Animation(
            // When used in string interpolation, the full case name is still used.
            // For 'castle' this will be 'buildings/castle'.
            name: "buildings/\(kind)",
            frameCount: frameCount,
            frameDuration: frameDuration
        )
    }
}

#52 Expressively comparing a value with a list of candidates

👨‍🔬 Continuing to experiment with expressive ways of comparing a value with a list of candidates in Swift. Adding an extension on Equatable is probably my favorite approach so far.

extension Equatable {
    func isAny(of candidates: Self...) -> Bool {
        return candidates.contains(self)
    }
}

let isHorizontal = direction.isAny(of: .left, .right)

See tip #35 for my previous experiment.

#51 UIView bounds and transforms

📐 A really interesting side-effect of a UIView's bounds being its rect within its own coordinate system is that transforms don't affect it at all. That's why it's usually a better fit than frame when doing layout calculations of subviews.

let view = UIView()
view.frame.size = CGSize(width: 100, height: 100)
view.transform = CGAffineTransform(scaleX: 2, y: 2)

print(view.frame) // (-50.0, -50.0, 200.0, 200.0)
print(view.bounds) // (0.0, 0.0, 100.0, 100.0)

#50 UIKit default arguments

👏 It's awesome that many UIKit APIs with completion handlers and other optional parameters import into Swift with default arguments (even though they are written in Objective-C). Getting rid of all those nil arguments is so nice!

// BEFORE: All parameters are specified, just like in Objective-C

viewController.present(modalViewController, animated: true, completion: nil)

modalViewController.dismiss(animated: true, completion: nil)

viewController.transition(from: loadingViewController,
                          to: contentViewController,
                          duration: 0.3,
                          options: [],
                          animations: animations,
                          completion: nil)

// AFTER: Since many UIKit APIs with completion handlers and other
// optional parameters import into Swift with default arguments,
// we can make our calls shorter

viewController.present(modalViewController, animated: true)

modalViewController.dismiss(animated: true)

viewController.transition(from: loadingViewController,
                          to: contentViewController,
                          duration: 0.3,
                          animations: animations)

#49 Avoiding Massive View Controllers

✂️ Avoiding Massive View Controllers is all about finding the right levels of abstraction and splitting things up.

My personal rule of thumb is that as soon as I have 3 methods or properties that have the same prefix, I break them out into their own type.

// BEFORE

class LoginViewController: UIViewController {
    private lazy var signUpLabel = UILabel()
    private lazy var signUpImageView = UIImageView()
    private lazy var signUpButton = UIButton()
}

// AFTER

class LoginViewController: UIViewController {
    private lazy var signUpView = SignUpView()
}

class SignUpView: UIView {
    private lazy var label = UILabel()
    private lazy var imageView = UIImageView()
    private lazy var button = UIButton()
}

#48 Extending optionals

❤️ I love the fact that optionals are enums in Swift - it makes it so easy to extend them with convenience APIs for certain types. Especially useful when doing things like data validation on optional values.

func validateTextFields() -> Bool {
    guard !usernameTextField.text.isNilOrEmpty else {
        return false
    }

    ...

    return true
}

// Since all optionals are actual enum values in Swift, we can easily
// extend them for certain types, to add our own convenience APIs

extension Optional where Wrapped == String {
    var isNilOrEmpty: Bool {
        switch self {
        case let string?:
            return string.isEmpty
        case nil:
            return true
        }
    }
}

// Since strings are now Collections in Swift 4, you can even
// add this property to all optional collections:

extension Optional where Wrapped: Collection {
    var isNilOrEmpty: Bool {
        switch self {
        case let collection?:
            return collection.isEmpty
        case nil:
            return true
        }
    }
}

#47 Using where with for-loops

🗺 Using the where keyword can be a super nice way to quickly apply a filter in a for-loop in Swift. You can of course use map, filter and forEach, or guard, but for simple loops I think this is very expressive and nice.

func archiveMarkedPosts() {
    for post in posts where post.isMarked {
        archive(post)
    }
}

func healAllies() {
    for player in players where player.isAllied(to: currentPlayer) {
        player.heal()
    }
}

#46 Variable shadowing

👻 Variable shadowing can be super useful in Swift, especially when you want to create a local copy of a parameter value in order to use it as state within a closure.

init(repeatMode: RepeatMode, closure: @escaping () -> UpdateOutcome) {
    // Shadow the argument with a local, mutable copy
    var repeatMode = repeatMode
    
    self.closure = {
        // With shadowing, there's no risk of accidentially
        // referring to the immutable version
        switch repeatMode {
        case .forever:
            break
        case .times(let count):
            guard count > 0 else {
                return .finished
            }
            
            // We can now capture the mutable version and use
            // it for state in a closure
            repeatMode = .times(count - 1)
        }
        
        return closure()
    }
}

#45 Using dot syntax for static properties and initializers

✒️ Dot syntax is one of my favorite features of Swift. What's really cool is that it's not only for enums, any static method or property can be used with dot syntax - even initializers! Perfect for convenience APIs and default parameters.

public enum RepeatMode {
    case times(Int)
    case forever
}

public extension RepeatMode {
    static var never: RepeatMode {
        return .times(0)
    }

    static var once: RepeatMode {
        return .times(1)
    }
}

view.perform(animation, repeated: .once)

// To make default parameters more compact, you can even use init with dot syntax

class ImageLoader {
    init(cache: Cache = .init(), decoder: ImageDecoder = .init()) {
        ...
    }
}

#44 Calling functions as closures with a tuple as parameters

🚀 One really cool aspect of Swift having first class functions is that you can pass any function (or even initializer) as a closure, and even call it with a tuple containing its parameters!

// This function lets us treat any "normal" function or method as
// a closure and run it with a tuple that contains its parameters
func call<Input, Output>(_ function: (Input) -> Output, with input: Input) -> Output {
    return function(input)
}

class ViewFactory {
    func makeHeaderView() -> HeaderView {
        // We can now pass an initializer as a closure, and a tuple
        // containing its parameters
        return call(HeaderView.init, with: loadTextStyles())
    }
    
    private func loadTextStyles() -> (font: UIFont, color: UIColor) {
        return (theme.font, theme.textColor)
    }
}

class HeaderView {
    init(font: UIFont, textColor: UIColor) {
        ...
    }
}

#43 Enabling static dependency injection

💉 If you've been struggling to test code that uses static APIs, here's a technique you can use to enable static dependency injection without having to modify any call sites:

// Before: Almost impossible to test due to the use of singletons

class Analytics {
    static func log(_ event: Event) {
        Database.shared.save(event)
        
        let dictionary = event.serialize()
        NetworkManager.shared.post(dictionary, to: eventURL)
    }
}

// After: Much easier to test, since we can inject mocks as arguments

class Analytics {
    static func log(_ event: Event,
                    database: Database = .shared,
                    networkManager: NetworkManager = .shared) {
        database.save(event)
        
        let dictionary = event.serialize()
        networkManager.post(dictionary, to: eventURL)
    }
}

#42 Type inference for lazy properties in Swift 4

🎉 In Swift 4, type inference works for lazy properties and you don't need to explicitly refer to self!

// Swift 3

class PurchaseView: UIView {
    private lazy var buyButton: UIButton = self.makeBuyButton()
    
    private func makeBuyButton() -> UIButton {
        let button = UIButton()
        button.setTitle("Buy", for: .normal)
        button.setTitleColor(.blue, for: .normal)
        return button
    }
}

// Swift 4

class PurchaseView: UIView {
    private lazy var buyButton = makeBuyButton()
    
    private func makeBuyButton() -> UIButton {
        let button = UIButton()
        button.setTitle("Buy", for: .normal)
        button.setTitleColor(.blue, for: .normal)
        return button
    }
}

#41 Converting Swift errors to NSError

😎 You can turn any Swift Error into an NSError, which is super useful when pattern matching with a code 👍. Also, switching on optionals is pretty cool!

let task = urlSession.dataTask(with: url) { data, _, error in
    switch error {
    case .some(let error as NSError) where error.code == NSURLErrorNotConnectedToInternet:
        presenter.showOfflineView()
    case .some(let error):
        presenter.showGenericErrorView()
    case .none:
        presenter.renderContent(from: data)
    }
}

task.resume()

Also make sure to check out Kostas Kremizas' tip about how you can pattern match directly against a member of URLError.

#40 Making UIImage macOS compatible

🖥 Here's an easy way to make iOS model code that uses UIImage macOS compatible - like me and Gui Rambo discussed on the Swift by Sundell Podcast.

// Either put this in a separate file that you only include in your macOS target or wrap the code in #if os(macOS) / #endif

import Cocoa

// Step 1: Typealias UIImage to NSImage
typealias UIImage = NSImage

// Step 2: You might want to add these APIs that UIImage has but NSImage doesn't.
extension NSImage {
    var cgImage: CGImage? {
        var proposedRect = CGRect(origin: .zero, size: size)

        return cgImage(forProposedRect: &proposedRect,
                       context: nil,
                       hints: nil)
    }

    convenience init?(named name: String) {
        self.init(named: Name(name))
    }
}

// Step 3: Profit - you can now make your model code that uses UIImage cross-platform!
struct User {
    let name: String
    let profileImage: UIImage
}

#39 Internally mutable protocol-oriented APIs

🤖 You can easily define a protocol-oriented API that can only be mutated internally, by using an internal protocol that extends a public one.

// Declare a public protocol that acts as your immutable API
public protocol ModelHolder {
    associatedtype Model
    var model: Model { get }
}

// Declare an extended, internal protocol that provides a mutable API
internal protocol MutableModelHolder: ModelHolder {
    var model: Model { get set }
}

// You can now implement the requirements using 'public internal(set)'
public class UserHolder: MutableModelHolder {
    public internal(set) var model: User

    internal init(model: User) {
        self.model = model
    }
}

#38 Switching on a set

🎛 You can switch on a set using array literals as cases in Swift! Can be really useful to avoid many if/else if statements.

class RoadTile: Tile {
    var connectedDirections = Set<Direction>()

    func render() {
        switch connectedDirections {
        case [.up, .down]:
            image = UIImage(named: "road-vertical")
        case [.left, .right]:
            image = UIImage(named: "road-horizontal")
        default:
            image = UIImage(named: "road")
        }
    }
}

#37 Adding the current locale to cache keys

🌍 When caching localized content in an app, it's a good idea to add the current locale to all keys, to prevent bugs when switching languages.

func cache(_ content: Content, forKey key: String) throws {
    let data = try wrap(content) as Data
    let key = localize(key: key)
    try storage.store(data, forKey: key)
}

func loadCachedContent(forKey key: String) -> Content? {
    let key = localize(key: key)
    let data = storage.loadData(forKey: key)
    return data.flatMap { try? unbox(data: $0) }
}

private func localize(key: String) -> String {
    return key + "-" + Bundle.main.preferredLocalizations[0]
}

#36 Setting up tests to avoid retain cycles with weak references

🚳 Here's an easy way to setup a test to avoid accidental retain cycles with object relationships (like weak delegates & observers) in Swift:

func testDelegateNotRetained() {
    // Assign the delegate (weak) and also retain it using a local var
    var delegate: Delegate? = DelegateMock()
    controller.delegate = delegate
    XCTAssertNotNil(controller.delegate)
    
    // Release the local var, which should also release the weak reference
    delegate = nil
    XCTAssertNil(controller.delegate)
}

#35 Expressively matching a value against a list of candidates

👨‍🔬 Playing around with an expressive way to check if a value matches any of a list of candidates in Swift:

// Instead of multiple conditions like this:

if string == "One" || string == "Two" || string == "Three" {

}

// You can now do:

if string == any(of: "One", "Two", "Three") {

}

You can find a gist with the implementation here.

#34 Organizing code using extensions

👪 APIs in a Swift extension automatically inherit its access control level, making it a neat way to organize public, internal & private APIs.

public extension Animation {
    init(textureNamed textureName: String) {
        frames = [Texture(name: textureName)]
    }
    
    init(texturesNamed textureNames: [String], frameDuration: TimeInterval = 1) {
        frames = textureNames.map(Texture.init)
        self.frameDuration = frameDuration
    }
    
    init(image: Image) {
        frames = [Texture(image: image)]
    }
}

internal extension Animation {
    func loadFrameImages() -> [Image] {
        return frames.map { $0.loadImageIfNeeded() }
    }
}

#33 Using map to transform an optional into a Result type

🗺 Using map you can transform an optional value into an optional Result type by simply passing in the enum case.

enum Result<Value> {
    case value(Value)
    case error(Error)
}

class Promise<Value> {
    private var result: Result<Value>?
    
    init(value: Value? = nil) {
        result = value.map(Result.value)
    }
}

#32 Assigning to self in struct initializers

👌 It's so nice that you can assign directly to self in struct initializers in Swift. Very useful when adding conformance to protocols.

extension Bool: AnswerConvertible {
    public init(input: String) throws {
        switch input.lowercased() {
        case "y", "yes", "👍":
            self = true
        default:
            self = false
        }
    }
}

#31 Recursively calling closures as inline functions

☎️ Defining Swift closures as inline functions enables you to recursively call them, which is super useful in things like custom sequences.

class Database {
    func records(matching query: Query) -> AnySequence<Record> {
        var recordIterator = loadRecords().makeIterator()
        
        func iterate() -> Record? {
            guard let nextRecord = recordIterator.next() else {
                return nil
            }
            
            guard nextRecord.matches(query) else {
                // Since the closure is an inline function, it can be recursively called,
                // in this case in order to advance to the next item.
                return iterate()
            }
            
            return nextRecord
        }
        
        // AnySequence/AnyIterator are part of the standard library and provide an easy way
        // to define custom sequences using closures.
        return AnySequence { AnyIterator(iterate) }
    }
}

Rob Napier points out that using the above might cause crashes if used on a large databaset, since Swift has no guaranteed Tail Call Optimization (TCO).

Slava Pestov also points out that another benefit of inline functions vs closures is that they can have their own generic parameter list.

#30 Passing self to required Objective-C dependencies

🏖 Using lazy properties in Swift, you can pass self to required Objective-C dependencies without having to use force-unwrapped optionals.

class DataLoader: NSObject {
    lazy var urlSession: URLSession = self.makeURLSession()
    
    private func makeURLSession() -> URLSession {
        return URLSession(configuration: .default, delegate: self, delegateQueue: .main)
    }
}

class Renderer {
    lazy var displayLink: CADisplayLink = self.makeDisplayLink()
    
    private func makeDisplayLink() -> CADisplayLink {
        return CADisplayLink(target: self, selector: #selector(screenDidRefresh))
    }
}

#29 Making weak or lazy properties readonly

👓 If you have a property in Swift that needs to be weak or lazy, you can still make it readonly by using private(set).

class Node {
    private(set) weak var parent: Node?
    private(set) lazy var children = [Node]()

    func add(child: Node) {
        children.append(child)
        child.parent = self
    }
}

#28 Defining static URLs using string literals

🌏 Tired of using URL(string: "url")! for static URLs? Make URL conform to ExpressibleByStringLiteral and you can now simply use "url" instead.

extension URL: ExpressibleByStringLiteral {
    // By using 'StaticString' we disable string interpolation, for safety
    public init(stringLiteral value: StaticString) {
        self = URL(string: "\(value)").require(hint: "Invalid URL string literal: \(value)")
    }
}

// We can now define URLs using static string literals 🎉
let url: URL = "https://www.swiftbysundell.com"
let task = URLSession.shared.dataTask(with: "https://www.swiftbysundell.com")

// In Swift 3 or earlier, you also have to implement 2 additional initializers
extension URL {
    public init(extendedGraphemeClusterLiteral value: StaticString) {
        self.init(stringLiteral: value)
    }

    public init(unicodeScalarLiteral value: StaticString) {
        self.init(stringLiteral: value)
    }
}

To find the extension that adds the require() method on Optional that I use above, check out Require.

#27 Manipulating points, sizes and frames using math operators

✚ I'm always careful with operator overloading, but for manipulating things like sizes, points & frames I find them super useful.

extension CGSize {
    static func *(lhs: CGSize, rhs: CGFloat) -> CGSize {
        return CGSize(width: lhs.width * rhs, height: lhs.height * rhs)
    }
}

button.frame.size = image.size * 2

If you like the above idea, check out CGOperators, which contains math operator overloads for all Core Graphics' vector types.

#26 Using closure types in generic constraints

🔗 You can use closure types in generic constraints in Swift. Enables nice APIs for handling sequences of closures.

extension Sequence where Element == () -> Void {
    func callAll() {
        forEach { $0() }
    }
}

extension Sequence where Element == () -> String {
    func joinedResults(separator: String) -> String {
        return map { $0() }.joined(separator: separator)
    }
}

callbacks.callAll()
let names = nameProviders.joinedResults(separator: ", ")

(If you're using Swift 3, you have to change Element to Iterator.Element)

#25 Using associated enum values to avoid state-specific optionals

🎉 Using associated enum values is a super nice way to encapsulate mutually exclusive state info (and avoiding state-specific optionals).

// BEFORE: Lots of state-specific, optional properties

class Player {
    var isWaitingForMatchMaking: Bool
    var invitingUser: User?
    var numberOfLives: Int
    var playerDefeatedBy: Player?
    var roundDefeatedIn: Int?
}

// AFTER: All state-specific information is encapsulated in enum cases

class Player {
    enum State {
        case waitingForMatchMaking
        case waitingForInviteResponse(from: User)
        case active(numberOfLives: Int)
        case defeated(by: Player, roundNumber: Int)
    }
    
    var state: State
}

#24 Using enums for async result types

👍 I really like using enums for all async result types, even boolean ones. Self-documenting, and makes the call site a lot nicer to read too!

protocol PushNotificationService {
    // Before
    func enablePushNotifications(completionHandler: @escaping (Bool) -> Void)
    
    // After
    func enablePushNotifications(completionHandler: @escaping (PushNotificationStatus) -> Void)
}

enum PushNotificationStatus {
    case enabled
    case disabled
}

service.enablePushNotifications { status in
    if status == .enabled {
        enableNotificationsButton.removeFromSuperview()
    }
}

#23 Working on async code in a playground

🏃 Want to work on your async code in a Swift Playground? Just set needsIndefiniteExecution to true to keep it running:

import PlaygroundSupport

PlaygroundPage.current.needsIndefiniteExecution = true

DispatchQueue.main.asyncAfter(deadline: .now() + 3) {
    let greeting = "Hello after 3 seconds"
    print(greeting)
}

To stop the playground from executing, simply call PlaygroundPage.current.finishExecution().

#22 Overriding self with a weak reference

💦 Avoid memory leaks when accidentially refering to self in closures by overriding it locally with a weak reference:

Swift >= 4.2

dataLoader.loadData(from: url) { [weak self] result in
    guard let self = self else { 
        return 
    }

    self.cache(result)
    
    ...

Swift < 4.2

dataLoader.loadData(from: url) { [weak self] result in
    guard let `self` = self else {
        return
    }

    self.cache(result)
    
    ...

Note that the reason the above currently works is because of a compiler bug (which I hope gets turned into a properly supported feature soon).

#21 Using DispatchWorkItem

🕓 Using dispatch work items you can easily cancel a delayed asynchronous GCD task if you no longer need it:

let workItem = DispatchWorkItem {
    // Your async code goes in here
}

// Execute the work item after 1 second
DispatchQueue.main.asyncAfter(deadline: .now() + 1, execute: workItem)

// You can cancel the work item if you no longer need it
workItem.cancel()

#20 Combining a sequence of functions

➕ While working on a new Swift developer tool (to be open sourced soon 😉), I came up with a pretty neat way of organizing its sequence of operations, by combining their functions into a closure:

internal func +<A, B, C>(lhs: @escaping (A) throws -> B,
                         rhs: @escaping (B) throws -> C) -> (A) throws -> C {
    return { try rhs(lhs($0)) }
}

public func run() throws {
    try (determineTarget + build + analyze + output)()
}

If you're familiar with the functional programming world, you might know the above technique as the pipe operator (thanks to Alexey Demedreckiy for pointing this out!)

#19 Chaining optionals with map() and flatMap()

🗺 Using map() and flatMap() on optionals you can chain multiple operations without having to use lengthy if lets or guards:

// BEFORE

guard let string = argument(at: 1) else {
    return
}

guard let url = URL(string: string) else {
    return
}

handle(url)

// AFTER

argument(at: 1).flatMap(URL.init).map(handle)

#18 Using self-executing closures for lazy properties

🚀 Using self-executing closures is a great way to encapsulate lazy property initialization:

class StoreViewController: UIViewController {
    private lazy var collectionView: UICollectionView = {
        let layout = UICollectionViewFlowLayout()
        let view = UICollectionView(frame: self.view.bounds, collectionViewLayout: layout)
        view.delegate = self
        view.dataSource = self
        return view
    }()
    
    override func viewDidLoad() {
        super.viewDidLoad()
        view.addSubview(collectionView)
    }
}

#17 Speeding up Swift package tests

⚡️ You can speed up your Swift package tests using the --parallel flag. For Marathon, the tests execute 3 times faster that way!

swift test --parallel

#16 Avoiding mocking UserDefaults

🛠 Struggling with mocking UserDefaults in a test? The good news is: you don't need mocking - just create a real instance:

class LoginTests: XCTestCase {
    private var userDefaults: UserDefaults!
    private var manager: LoginManager!
    
    override func setUp() {
        super.setup()
        
        userDefaults = UserDefaults(suiteName: #file)
        userDefaults.removePersistentDomain(forName: #file)
        
        manager = LoginManager(userDefaults: userDefaults)
    }
}

#15 Using variadic parameters

👍 Using variadic parameters in Swift, you can create some really nice APIs that take a list of objects without having to use an array:

extension Canvas {
    func add(_ shapes: Shape...) {
        shapes.forEach(add)
    }
}

let circle = Circle(center: CGPoint(x: 5, y: 5), radius: 5)
let lineA = Line(start: .zero, end: CGPoint(x: 10, y: 10))
let lineB = Line(start: CGPoint(x: 0, y: 10), end: CGPoint(x: 10, y: 0))

let canvas = Canvas()
canvas.add(circle, lineA, lineB)
canvas.render()

#14 Referring to enum cases with associated values as closures

😮 Just like you can refer to a Swift function as a closure, you can do the same thing with enum cases with associated values:

enum UnboxPath {
    case key(String)
    case keyPath(String)
}

struct UserSchema {
    static let name = key("name")
    static let age = key("age")
    static let posts = key("posts")
    
    private static let key = UnboxPath.key
}

#13 Using the === operator to compare objects by instance

📈 The === operator lets you check if two objects are the same instance. Very useful when verifying that an array contains an instance in a test:

protocol InstanceEquatable: class, Equatable {}

extension InstanceEquatable {
    static func ==(lhs: Self, rhs: Self) -> Bool {
        return lhs === rhs
    }
}

extension Enemy: InstanceEquatable {}

func testDestroyingEnemy() {
    player.attack(enemy)
    XCTAssertTrue(player.destroyedEnemies.contains(enemy))
}

#12 Calling initializers with dot syntax and passing them as closures

😎 Cool thing about Swift initializers: you can call them using dot syntax and pass them as closures! Perfect for mocking dates in tests.

class Logger {
    private let storage: LogStorage
    private let dateProvider: () -> Date
    
    init(storage: LogStorage = .init(), dateProvider: @escaping () -> Date = Date.init) {
        self.storage = storage
        self.dateProvider = dateProvider
    }
    
    func log(event: Event) {
        storage.store(event: event, date: dateProvider())
    }
}

#11 Structuring UI tests as extensions on XCUIApplication

📱 Most of my UI testing logic is now categories on XCUIApplication. Makes the test cases really easy to read:

func testLoggingInAndOut() {
    XCTAssertFalse(app.userIsLoggedIn)
    
    app.launch()
    app.login()
    XCTAssertTrue(app.userIsLoggedIn)
    
    app.logout()
    XCTAssertFalse(app.userIsLoggedIn)
}

func testDisplayingCategories() {
    XCTAssertFalse(app.isDisplayingCategories)
    
    app.launch()
    app.login()
    app.goToCategories()
    XCTAssertTrue(app.isDisplayingCategories)
}

#10 Avoiding default cases in switch statements

🙂 It’s a good idea to avoid “default” cases when switching on Swift enums - it’ll “force you” to update your logic when a new case is added:

enum State {
    case loggedIn
    case loggedOut
    case onboarding
}

func handle(_ state: State) {
    switch state {
    case .loggedIn:
        showMainUI()
    case .loggedOut:
        showLoginUI()
    // Compiler error: Switch must be exhaustive
    }
}

#9 Using the guard statement in many different scopes

💂 It's really cool that you can use Swift's 'guard' statement to exit out of pretty much any scope, not only return from functions:

// You can use the 'guard' statement to...

for string in strings {
    // ...continue an iteration
    guard shouldProcess(string) else {
        continue
    }
    
    // ...or break it
    guard !shouldBreak(for: string) else {
        break
    }
    
    // ...or return
    guard !shouldReturn(for: string) else {
        return
    }
    
    // ..or throw an error
    guard string.isValid else {
        throw StringError.invalid(string)
    }
    
    // ...or exit the program
    guard !shouldExit(for: string) else {
        exit(1)
    }
}

#8 Passing functions & operators as closures

❤️ Love how you can pass functions & operators as closures in Swift. For example, it makes the syntax for sorting arrays really nice!

let array = [3, 9, 1, 4, 6, 2]
let sorted = array.sorted(by: <)

#7 Using #function for UserDefaults key consistency

🗝 Here's a neat little trick I use to get UserDefault key consistency in Swift (#function expands to the property name in getters/setters). Just remember to write a good suite of tests that'll guard you against bugs when changing property names.

extension UserDefaults {
    var onboardingCompleted: Bool {
        get { return bool(forKey: #function) }
        set { set(newValue, forKey: #function) }
    }
}

#6 Using a name already taken by the standard library

📛 Want to use a name already taken by the standard library for a nested type? No problem - just use Swift. to disambiguate:

extension Command {
    enum Error: Swift.Error {
        case missing
        case invalid(String)
    }
}

#5 Using Wrap to implement Equatable

📦 Playing around with using Wrap to implement Equatable for any type, primarily for testing:

protocol AutoEquatable: Equatable {}

extension AutoEquatable {
    static func ==(lhs: Self, rhs: Self) -> Bool {
        let lhsData = try! wrap(lhs) as Data
        let rhsData = try! wrap(rhs) as Data
        return lhsData == rhsData
    }
}

#4 Using typealiases to reduce the length of method signatures

📏 One thing that I find really useful in Swift is to use typealiases to reduce the length of method signatures in generic types:

public class PathFinder<Object: PathFinderObject> {
    public typealias Map = Object.Map
    public typealias Node = Map.Node
    public typealias Path = PathFinderPath<Object>
    
    public static func possiblePaths(for object: Object, at rootNode: Node, on map: Map) -> Path.Sequence {
        return .init(object: object, rootNode: rootNode, map: map)
    }
}

#3 Referencing either external or internal parameter name when writing docs

📖 You can reference either the external or internal parameter label when writing Swift docs - and they get parsed the same:

// EITHER:

class Foo {
    /**
    *   - parameter string: A string
    */
    func bar(with string: String) {}
}

// OR:

class Foo {
    /**
    *   - parameter with: A string
    */
    func bar(with string: String) {}
}

#2 Using auto closures

👍 Finding more and more uses for auto closures in Swift. Can enable some pretty nice APIs:

extension Dictionary {
    mutating func value(for key: Key, orAdd valueClosure: @autoclosure () -> Value) -> Value {
        if let value = self[key] {
            return value
        }
        
        let value = valueClosure()
        self[key] = value
        return value
    }
}

#1 Namespacing with nested types

🚀 I’ve started to become a really big fan of nested types in Swift. Love the additional namespacing it gives you!

public struct Map {
    public struct Model {
        public let size: Size
        public let theme: Theme
        public var terrain: [Position : Terrain.Model]
        public var units: [Position : Unit.Model]
        public var buildings: [Position : Building.Model]
    }
    
    public enum Direction {
        case up
        case right
        case down
        case left
    }
    
    public struct Position {
        public var x: Int
        public var y: Int
    }
    
    public enum Size: String {
        case small = "S"
        case medium = "M"
        case large = "L"
        case extraLarge = "XL"
    }
}

---

Download Details:

Author: JohnSundell
Source code: https://github.com/JohnSundell/SwiftTips

License: MIT license
#swift 

Socket: Non-blocking socket and TLS functionality for PHP based on Amp

Socket

amphp/socket is a socket library for establishing and encrypting non-blocking sockets PHP based on Amp.

Installation

This package can be installed as a Composer dependency.

composer require amphp/socket

Documentation

Documentation can be found on amphp.org as well as in the ./docs directory.

Examples

You can find more examples in the ./examples directory.

Client Example

<?php // basic (and dumb) HTTP client

require __DIR__ . '/../vendor/autoload.php';

// This is a very simple HTTP client that just prints the response without parsing.
// league/uri-schemes required for this example.

use Amp\ByteStream;
use Amp\Loop;
use Amp\Socket\ClientTlsContext;
use Amp\Socket\ConnectContext;
use Amp\Socket\EncryptableSocket;
use League\Uri;
use function Amp\Socket\connect;

Loop::run(static function () use ($argv) {
    $stdout = ByteStream\getStdout();

    if (\count($argv) !== 2) {
        yield $stdout->write('Usage: examples/simple-http-client.php <url>' . PHP_EOL);
        exit(1);
    }

    $uri = Uri\Http::createFromString($argv[1]);
    $host = $uri->getHost();
    $port = $uri->getPort() ?? ($uri->getScheme() === 'https' ? 443 : 80);
    $path = $uri->getPath() ?: '/';

    $connectContext = (new ConnectContext)
        ->withTlsContext(new ClientTlsContext($host));

    /** @var EncryptableSocket $socket */
    $socket = yield connect($host . ':' . $port, $connectContext);

    if ($uri->getScheme() === 'https') {
        yield $socket->setupTls();
    }

    yield $socket->write("GET {$path} HTTP/1.1\r\nHost: $host\r\nConnection: close\r\n\r\n");

    while (null !== $chunk = yield $socket->read()) {
        yield $stdout->write($chunk);
    }

    // If the promise returned from `read()` resolves to `null`, the socket closed and we're done.
    // In this case you can also use `yield Amp\ByteStream\pipe($socket, $stdout)` instead,
    // but we want to demonstrate the `read()` method here.
});

Server Example

<?php // basic (and dumb) HTTP server

require __DIR__ . '/../vendor/autoload.php';

// This is a very simple HTTP server that just prints a message to each client that connects.
// It doesn't check whether the client sent an HTTP request.

// You might notice that your browser opens several connections instead of just one,
// even when only making one request.

use Amp\Loop;
use Amp\Socket\ResourceSocket;
use Amp\Socket\Server;
use function Amp\asyncCoroutine;

Loop::run(static function () {
    $clientHandler = asyncCoroutine(static function (ResourceSocket $socket) {
        $address = $socket->getRemoteAddress();
        $ip = $address->getHost();
        $port = $address->getPort();

        echo "Accepted connection from {$address}." . PHP_EOL;

        $body = "Hey, your IP is {$ip} and your local port used is {$port}.";
        $bodyLength = \strlen($body);

        $req = "HTTP/1.1 200 OK\r\nConnection: close\r\nContent-Length: {$bodyLength}\r\n\r\n{$body}";
        yield $socket->end($req);
    });

    $server = Server::listen('127.0.0.1:0');

    echo 'Listening for new connections on ' . $server->getAddress() . ' ...' . PHP_EOL;
    echo 'Open your browser and visit http://' . $server->getAddress() . '/' . PHP_EOL;

    while ($socket = yield $server->accept()) {
        $clientHandler($socket);
    }
});

Security

If you discover any security related issues, please email me@kelunik.com instead of using the issue tracker.

Download Details:

Author: Amphp
Source Code: https://github.com/amphp/socket 
License: MIT license

#php #client #socket #unix 

How To Secure A Linux Server

An evolving how-to guide for securing a Linux server that, hopefully, also teaches you a little about security and why it matters.

Introduction

Guide Objective

This guides purpose is to teach you how to secure a Linux server.

There are a lot of things you can do to secure a Linux server and this guide will attempt to cover as many of them as possible. More topics/material will be added as I learn, or as folks contribute.

(Table of Contents)

Why Secure Your Server

I assume you're using this guide because you, hopefully, already understand why good security is important. That is a heavy topic onto itself and breaking it down is out-of-scope for this guide. If you don't know the answer to that question, I advise you research it first.

At a high level, the second a device, like a server, is in the public domain -- i.e visible to the outside world -- it becomes a target for bad-actors. An unsecured device is a playground for bad-actors who want access to your data, or to use your server as another node for their large-scale DDOS attacks.

What's worse is, without good security, you may never know if your server has been compromised. A bad-actor may have gained unauthorized access to your server and copied your data without changing anything so you'd never know. Or your server may have been part of a DDOS attack and you wouldn't know. Look at many of the large scale data breaches in the news -- the companies often did not discover the data leak or intrusion until long after the bad-actors were gone.

Contrary to popular belief, bad-actors don't always want to change something or lock you out of your data for money. Sometimes they just want the data on your server for their data warehouses (there is big money in big data) or to covertly use your server for their nefarious purposes.

(Table of Contents)

Why Yet Another Guide

This guide may appear duplicative/unnecessary because there are countless articles online that tell you how to secure Linux, but the information is spread across different articles, that cover different things, and in different ways. Who has time to scour through hundreds of articles?

As I was going through research for my Debian build, I kept notes. At the end I realized that, along with what I already knew, and what I was learning, I had the makings of a how-to guide. I figured I'd put it online to hopefully help others learn, and save time.

I've never found one guide that covers everything -- this guide is my attempt.

Many of the things covered in this guide may be rather basic/trivial, but most of us do not install Linux every day and it is easy to forget those basic things.

IT automation tools like Ansible, Chef, Jenkins, Puppet, etc. help with the tedious task of installing/configuring a server but IMHO they are better suited for multiple or large scale deployments. IMHO, the overhead required to use those kinds of automation tools is wholly unnecessary for a one-time single server install for home use.

Other Guides

There are many guides provided by experts, industry leaders, and the distributions themselves. It is not practical, and sometimes against copyright, to include everything from those guides. I recommend you check them out before starting with this guide.

• The Center for Internet Security (CIS) provides benchmarks that are exhaustive, industry trusted, step-by-step instructions for securing many flavors of Linux. Check their About Us page for details. My recommendation is to go through this guide first and then CIS's guide. That way their recommendations will trump anything in this guide.
• For distribution specific hardening/security guides, check your distributions documentation.
• https://security.utexas.edu/os-hardening-checklist/linux-7 - Red Hat Enterprise Linux 7 Hardening Checklist
• https://cloudpro.zone/index.php/2018/01/18/debian-9-3-server-setup-guide-part-1/ - # Debian 9.3 server setup guide
• https://blog.vigilcode.com/2011/04/ubuntu-server-initial-security-quick-secure-setup-part-i/ - Ubuntu Server Initial Security guide
• https://www.tldp.org/LDP/sag/html/index.html
• https://seifried.org/lasg/
• https://news.ycombinator.com/item?id=19178964
• https://wiki.archlinux.org/index.php/Security - many folks have also recommended this one
• https://securecompliance.co/linux-server-hardening-checklist/

(Table of Contents)

To Do / To Add

• Custom Jails for Fail2ban
• MAC (Mandatory Access Control) and Linux Security Modules (LSMs)
  • https://wiki.archlinux.org/index.php/security#Mandatory_access_control
  • Security-Enhanced Linux / SELinux
    • https://en.wikipedia.org/wiki/Security-Enhanced_Linux
    • https://linuxtechlab.com/beginners-guide-to-selinux/
    • https://linuxtechlab.com/replicate-selinux-policies-among-linux-machines/
    • https://teamignition.us/how-to-stop-being-a-scrub-and-learn-to-use-selinux.html
  • AppArmor
    • https://wiki.archlinux.org/index.php/AppArmor
    • https://security.stackexchange.com/questions/29378/comparison-between-apparmor-and-selinux
    • http://www.insanitybit.com/2012/06/01/why-i-like-apparmor-more-than-selinux-5/
• disk encryption
• Rkhunter and chrootkit
  • http://www.chkrootkit.org/
  • http://rkhunter.sourceforge.net/
  • https://www.cyberciti.biz/faq/howto-check-linux-rootkist-with-detectors-software/
  • https://www.tecmint.com/install-rootkit-hunter-scan-for-rootkits-backdoors-in-linux/
• shipping/backing up logs - https://news.ycombinator.com/item?id=19178681
• CIS-CAT - https://learn.cisecurity.org/cis-cat-landing-page
• debsums - https://blog.sleeplessbeastie.eu/2015/03/02/how-to-verify-installed-packages/

Guide Overview

About This Guide

This guide...

• ...is a work in progress.
• ...is focused on at-home Linux servers. All of the concepts/recommendations here apply to larger/professional environments but those use-cases call for more advanced and specialized configurations that are out-of-scope for this guide.
• ...does not teach you about Linux, how to install Linux, or how to use it. Check https://linuxjourney.com/ if you're new to Linux.
• ...is meant to be Linux distribution agnostic.
• ...does not teach you everything you need to know about security nor does it get into all aspects of system/server security. For example, physical security is out of scope for this guide.
• ...does not talk about how programs/tools work, nor does it delve into their nook and crannies. Most of the programs/tools this guide references are very powerful and highly configurable. The goal is to cover the bare necessities -- enough to whet your appetite and make you hungry enough to want to go and learn more.
• ...aims to make it easy by providing code you can copy-and-paste. You might need to modify the commands before you paste so keep your favorite text editor handy.
• ...is organized in an order that makes logical sense to me -- i.e. securing SSH before installing a firewall. As such, this guide is intended to be followed in the order it is presented but it is not necessary to do so. Just be careful if you do things in a different order -- some sections require previous sections to be completed.

My Use-Case

There are many types of servers and different use-cases. While I want this guide to be as generic as possible, there will be some things that may not apply to all/other use-cases. Use your best judgement when going through this guide.

To help put context to many of the topics covered in this guide, my use-case/configuration is:

• A desktop class computer...
• With a single NIC...
• Connected to a consumer grade router...
• Getting a dynamic WAN IP provided by the ISP...
• With WAN+LAN on IPV4...
• And LAN using NAT...
• That I want to be able to SSH to remotely from unknown computers and unknown locations (i.e. a friend's house).

Editing Configuration Files - For The Lazy

I am very lazy and do not like to edit files by hand if I don't need to. I also assume everyone else is just like me. :)

So, when and where possible, I have provided code snippets to quickly do what is needed, like add or change a line in a configuration file.

The code snippets use basic commands like echo, cat, sed, awk, and grep. How the code snippets work, like what each command/part does, is out of scope for this guide -- the man pages are your friend.

Note: The code snippets do not validate/verify the change went through -- i.e. the line was actually added or changed. I'll leave the verifying part in your capable hands. The steps in this guide do include taking backups of all files that will be changed.

Not all changes can be automated with code snippets. Those changes need good, old fashioned, manual editing. For example, you can't just append a line to an INI type file. Use your favorite Linux text editor.

Contributing

I wanted to put this guide on GitHub to make it easy to collaborate. The more folks that contribute, the better and more complete this guide will become.

To contribute you can fork and submit a pull request or submit a new issue.

Before You Start

Identify Your Principles

Before you start you will want to identify what your Principles are. What is your threat model? Some things to think about:

• Why do you want to secure your server?
• How much security do you want or not want?
• How much convenience are you willing to compromise for security and vice-versa?
• What are the threats you want to protect against? What are the specifics to your situation? For example:
  • Is physical access to your server/network a possible attack vector?
  • Will you be opening ports on your router so you can access your server from outside your home?
  • Will you be hosting a file share on your server that will be mounted on a desktop class machine? What is the possibility of the desktop machine getting infected and, in turn, infecting the server?
• Do you have a means of recovering if your security implementation locks you out of your own server? For example, you disabled root login or password protected GRUB.

These are just a few things to think about. Before you start securing your server you will want to understand what you're trying to protect against and why so you know what you need to do.

Picking A Linux Distribution

This guide is intended to be distribution agnostic so users can use any distribution they want. With that said, there are a few things to keep in mind:

You want a distribution that...

• ...is stable. Unless you like debugging issues at 2 AM, you don't want an unattended upgrade, or a manual package/system update, to render your server inoperable. But this also means you're okay with not running the latest, greatest, bleeding edge software.
• ...stays up-to-date with security patches. You can secure everything on your server, but if the core OS or applications you're running have known vulnerabilities, you'll never be safe.
• ...you're familiar with. If you don't know Linux, I would advise you play around with one before you try to secure it. You should be comfortable with it and know your way around, like how to install software, where configuration files are, etc...
• ...is well supported. Even the most seasoned admin needs help every now and then. Having a place to go for help will save your sanity.

Installing Linux

Installing Linux is out-of-scope for this guide because each distribution does it differently and the installation instructions are usually well documented. If you need help, start with your distribution's documentation. Regardless of the distribution, the high-level process usually goes like so:

1. download the ISO
2. burn/copy/transfer it to your install medium (e.g. a CD or USB stick)
3. boot your server from your install medium
4. follow the prompts to install

Where applicable, use the expert install option so you have tighter control of what is running on your server. Only install what you absolutely need. I, personally, do not install anything other than SSH. Also, tick the Disk Encryption option.

Pre/Post Installation Requirements

• If you're opening ports on your router so you can access your server from the outside, disable the port forwarding until your system is up and secured.
• Unless you're doing everything physically connected to your server, you'll need remote access so be sure SSH works.
• Keep your system up-to-date (i.e. sudo apt update && sudo apt upgrade on Debian based systems).
• Make sure you perform any tasks specific to your setup like:
  • Configuring network
  • Configuring mount points in /etc/fstab
  • Creating the initial user accounts
  • Installing core software you'll want like man
  • Etc...
• Your server will need to be able to send e-mails so you can get important security alerts. If you're not setting up a mail server check Gmail and Exim4 As MTA With Implicit TLS.
• I would also recommend you go through the CIS Benchmarks before you start with this guide.

Other Important Notes

• This guide is being written and tested on Debian. Most things below should work on other distributions. If you find something that does not, please contact me. The main thing that separates each distribution will be its package management system. Since I use Debian, I will provide the appropriate apt commands that should work on all Debian based distributions. If someone is willing to provide the respective commands for other distributions, I will add them.
• File paths and settings also may differ slightly -- check with your distribution's documentation if you have issues.
• Read the whole guide before you start. Your use-case and/or principals may call for not doing something or for changing the order.
• Do not blindly copy-and-paste without understanding what you're pasting. Some commands will need to be modified for your needs before they'll work -- usernames for example.

The SSH Server

Important Note Before You Make SSH Changes

It is highly advised you keep a 2nd terminal open to your server before you make and apply SSH configuration changes. This way if you lock yourself out of your 1st terminal session, you still have one session connected so you can fix it.

Thank you to Sonnenbrand for this idea.

SSH Public/Private Keys

Why

Using SSH public/private keys is more secure than using a password. It also makes it easier and faster, to connect to our server because you don't have to enter a password.

How It Works

Check the references below for more details but, at a high level, public/private keys work by using a pair of keys to verify identity.

1. One key, the public key, can only encrypt data, not decrypt it
2. The other key, the private key, can decrypt the data

For SSH, a public and private key is created on the client. You want to keep both keys secure, especially the private key. Even though the public key is meant to be public, it is wise to make sure neither keys fall in the wrong hands.

When you connect to an SSH server, SSH will look for a public key that matches the client you're connecting from in the file ~/.ssh/authorized_keys on the server you're connecting to. Notice the file is in the home folder of the ID you're trying to connect to. So, after creating the public key, you need to append it to ~/.ssh/authorized_keys. One approach is to copy it to a USB stick and physically transfer it to the server. Another approach is to use use ssh-copy-id to transfer and append the public key.

After the keys have been created and the public key has been appended to ~/.ssh/authorized_keys on the host, SSH uses the public and private keys to verify identity and then establish a secure connection. How identity is verified is a complicated process but Digital Ocean has a very nice write-up of how it works. At a high level, identity is verified by the server encrypting a challenge message with the public key, then sending it to the client. If the client cannot decrypt the challenge message with the private key, the identity can't be verified and a connection will not be established.

They are considered more secure because you need the private key to establish an SSH connection. If you set PasswordAuthentication no in /etc/ssh/sshd_config, then SSH won't let you connect without the private key.

You can also set a pass-phrase for the keys which would require you to enter the key pass-phrase when connecting using public/private keys. Keep in mind doing this means you can't use the key for automation because you'll have no way to send the passphrase in your scripts. ssh-agent is a program that is shipped in many Linux distros (and usually already running) that will allow you to hold your unencrypted private key in memory for a configurable duration. Simply run ssh-add and it will prompt you for your passphrase. You will not be prompted for your passphrase again until the configurable duration has passed.

We will be using Ed25519 keys which, according to https://linux-audit.com/:

It is using an elliptic curve signature scheme, which offers better security than ECDSA and DSA. At the same time, it also has good performance.

Goals

• Ed25519 public/private SSH keys:
  • private key on your client
  • public key on your server

Notes

• You'll need to do this step for every computer and account you'll be connecting to your server from/as.

References

• https://www.ssh.com/ssh/public-key-authentication
• https://help.ubuntu.com/community/SSH/OpenSSH/Keys
• https://linux-audit.com/using-ed25519-openssh-keys-instead-of-dsa-rsa-ecdsa/
• https://www.digitalocean.com/community/tutorials/understanding-the-ssh-encryption-and-connection-process
• https://wiki.archlinux.org/index.php/SSH_Keys
• https://www.ssh.com/ssh/copy-id
• man ssh-keygen
• man ssh-copy-id
• man ssh-add

Steps

From the computer you're going to use to connect to your server, the client, not the server itself, create an Ed25519 key with ssh-keygen:

ssh-keygen -t ed25519

Generating public/private ed25519 key pair.
Enter file in which to save the key (/home/user/.ssh/id_ed25519):
Created directory '/home/user/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/user/.ssh/id_ed25519.
Your public key has been saved in /home/user/.ssh/id_ed25519.pub.
The key fingerprint is:
SHA256:F44D4dr2zoHqgj0i2iVIHQ32uk/Lx4P+raayEAQjlcs user@client
The key's randomart image is:
+--[ED25519 256]--+
|xxxx  x          |
|o.o +. .         |
| o o oo   .      |
|. E oo . o .     |
| o o. o S o      |
|... .. o o       |
|.+....+ o        |
|+.=++o.B..       |
|+..=**=o=.       |
+----[SHA256]-----+

Note: If you set a passphrase, you'll need to enter it every time you connect to your server using this key, unless you're using ssh-agent.

Now you need to append the public key ~/.ssh/id_ed25519.pub from your client to the ~/.ssh/authorized_keys file on your server. Since we're presumable still at home on the LAN, we're probably safe from MIM attacks, so we will use ssh-copy-id to transfer and append the public key:

ssh-copy-id user@server

/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/home/user/.ssh/id_ed25519.pub"
The authenticity of host 'host (192.168.1.96)' can't be established.
ECDSA key fingerprint is SHA256:QaDQb/X0XyVlogh87sDXE7MR8YIK7ko4wS5hXjRySJE.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
user@host's password:

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'user@host'"
and check to make sure that only the key(s) you wanted were added.

Now would be a good time to perform any tasks specific to your setup.

Create SSH Group For AllowGroups

Why

To make it easy to control who can SSH to the server. By using a group, we can quickly add/remove accounts to the group to quickly allow or not allow SSH access to the server.

How It Works

We will use the AllowGroups option in SSH's configuration file /etc/ssh/sshd_config to tell the SSH server to only allow users to SSH in if they are a member of a certain UNIX group. Anyone not in the group will not be able to SSH in.

Goals

• a UNIX group that we'll use in Secure /etc/ssh/sshd_config to limit who can SSH to the server

Notes

• This is a per-requisite step to support the AllowGroup setting set in Secure /etc/ssh/sshd_config.

References

• man groupadd
• man usermod

Steps

Create a group:

sudo groupadd sshusers

Add account(s) to the group:

sudo usermod -a -G sshusers user1
sudo usermod -a -G sshusers user2
sudo usermod -a -G sshusers ...

You'll need to do this for every account on your server that needs SSH access.

Secure /etc/ssh/sshd_config

Why

SSH is a door into your server. This is especially true if you are opening ports on your router so you can SSH to your server from outside your home network. If it is not secured properly, a bad-actor could use it to gain unauthorized access to your system.

How It Works

/etc/ssh/sshd_config is the default configuration file that the SSH server uses. We will use this file to tell what options the SSH server should use.

Goals

• a secure SSH configuration

Notes

• Make sure you've completed Create SSH Group For AllowGroups first.

References

• Mozilla's OpenSSH guidelines for OpenSSH 6.7+ at https://infosec.mozilla.org/guidelines/openssh#modern-openssh-67
• https://linux-audit.com/audit-and-harden-your-ssh-configuration/
• https://www.ssh.com/ssh/sshd_config/
• https://www.techbrown.com/harden-ssh-secure-linux-vps-server/
• https://serverfault.com/questions/660160/openssh-difference-between-internal-sftp-and-sftp-server/660325
• man sshd_config
• Thanks to than0s for how to find duplicate settings.

Steps

Make a backup of OpenSSH server's configuration file /etc/ssh/sshd_config and remove comments to make it easier to read:

sudo cp --archive /etc/ssh/sshd_config /etc/ssh/sshd_config-COPY-$(date +"%Y%m%d%H%M%S")
sudo sed -i -r -e '/^#|^$/ d' /etc/ssh/sshd_config

Edit /etc/ssh/sshd_config then find and edit or add these settings that should be applied regardless of your configuration/setup:

Note: SSH does not like duplicate contradicting settings. For example, if you have ChallengeResponseAuthentication no and then ChallengeResponseAuthentication yes, SSH will respect the first one and ignore the second. Your /etc/ssh/sshd_config file may already have some of the settings/lines below. To avoid issues you will need to manually go through your /etc/ssh/sshd_config file and address any duplicate contradicting settings.

########################################################################################################
# start settings from https://infosec.mozilla.org/guidelines/openssh#modern-openssh-67 as of 2019-01-01
########################################################################################################

# Supported HostKey algorithms by order of preference.
HostKey /etc/ssh/ssh_host_ed25519_key
HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_ecdsa_key

KexAlgorithms curve25519-sha256@libssh.org,ecdh-sha2-nistp521,ecdh-sha2-nistp384,ecdh-sha2-nistp256,diffie-hellman-group-exchange-sha256

Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com,aes256-ctr,aes192-ctr,aes128-ctr

MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512,hmac-sha2-256,umac-128@openssh.com

# LogLevel VERBOSE logs user's key fingerprint on login. Needed to have a clear audit track of which key was using to log in.
LogLevel VERBOSE

# Use kernel sandbox mechanisms where possible in unprivileged processes
# Systrace on OpenBSD, Seccomp on Linux, seatbelt on MacOSX/Darwin, rlimit elsewhere.
# Note: This setting is deprecated in OpenSSH 7.5 (https://www.openssh.com/txt/release-7.5)
# UsePrivilegeSeparation sandbox

########################################################################################################
# end settings from https://infosec.mozilla.org/guidelines/openssh#modern-openssh-67 as of 2019-01-01
########################################################################################################

# don't let users set environment variables
PermitUserEnvironment no

# Log sftp level file access (read/write/etc.) that would not be easily logged otherwise.
Subsystem sftp  internal-sftp -f AUTHPRIV -l INFO

# only use the newer, more secure protocol
Protocol 2

# disable X11 forwarding as X11 is very insecure
# you really shouldn't be running X on a server anyway
X11Forwarding no

# disable port forwarding
AllowTcpForwarding no
AllowStreamLocalForwarding no
GatewayPorts no
PermitTunnel no

# don't allow login if the account has an empty password
PermitEmptyPasswords no

# ignore .rhosts and .shosts
IgnoreRhosts yes

# verify hostname matches IP
UseDNS yes

Compression no
TCPKeepAlive no
AllowAgentForwarding no
PermitRootLogin no

# don't allow .rhosts or /etc/hosts.equiv
HostbasedAuthentication no

Then find and edit or add these settings, and set values as per your requirements:

Setting	Valid Values	Example	Description	Notes
AllowGroups	local UNIX group name	AllowGroups sshusers	group to allow SSH access to
ClientAliveCountMax	number	ClientAliveCountMax 0	maximum number of client alive messages sent without response
ClientAliveInterval	number of seconds	ClientAliveInterval 300	timeout in seconds before a response request
ListenAddress	space separated list of local addresses	ListenAddress 0.0.0.0 ListenAddress 192.168.1.100	local addresses sshd should listen on	See Issue #1 for important details.
LoginGraceTime	number of seconds	LoginGraceTime 30	time in seconds before login times-out
MaxAuthTries	number	MaxAuthTries 2	maximum allowed attempts to login
MaxSessions	number	MaxSessions 2	maximum number of open sessions
MaxStartups	number	MaxStartups 2	maximum number of login sessions
PasswordAuthentication	yes or no	PasswordAuthentication no	if login with a password is allowed
Port	any open/available port number	Port 22	port that sshd should listen on

Check man sshd_config for more details what these settings mean.

Make sure there are no duplicate settings that contradict each other. The below command should not have any output.

awk 'NF && $1!~/^(#|HostKey)/{print $1}' /etc/ssh/sshd_config | sort | uniq -c | grep -v ' 1 '

Restart ssh:

sudo service sshd restart

You can check verify the configurations worked with sshd -T and verify the output:

sudo sshd -T

port 22
addressfamily any
listenaddress [::]:22
listenaddress 0.0.0.0:22
usepam yes
logingracetime 30
x11displayoffset 10
maxauthtries 2
maxsessions 2
clientaliveinterval 300
clientalivecountmax 0
streamlocalbindmask 0177
permitrootlogin no
ignorerhosts yes
ignoreuserknownhosts no
hostbasedauthentication no
...
subsystem sftp internal-sftp -f AUTHPRIV -l INFO
maxstartups 2:30:2
permittunnel no
ipqos lowdelay throughput
rekeylimit 0 0
permitopen any

Remove Short Diffie-Hellman Keys

Why

Per Mozilla's OpenSSH guidelines for OpenSSH 6.7+, "all Diffie-Hellman moduli in use should be at least 3072-bit-long".

The Diffie-Hellman algorithm is used by SSH to establish a secure connection. The larger the moduli (key size) the stronger the encryption.

Goals

• remove all Diffie-Hellman keys that are less than 3072 bits long

References

• Mozilla's OpenSSH guidelines for OpenSSH 6.7+ at https://infosec.mozilla.org/guidelines/openssh#modern-openssh-67
• https://infosec.mozilla.org/guidelines/key_management
• man moduli

Steps

Make a backup of SSH's moduli file /etc/ssh/moduli:

sudo cp --archive /etc/ssh/moduli /etc/ssh/moduli-COPY-$(date +"%Y%m%d%H%M%S")

Remove short moduli:

sudo awk '$5 >= 3071' /etc/ssh/moduli | sudo tee /etc/ssh/moduli.tmp
sudo mv /etc/ssh/moduli.tmp /etc/ssh/moduli

2FA/MFA for SSH

Why

Even though SSH is a pretty good security guard for your doors and windows, it is still a visible door that bad-actors can see and try to brute-force in. Fail2ban will monitor for these brute-force attempts but there is no such thing as being too secure. Requiring two factors adds an extra layer of security.

Using Two Factor Authentication (2FA) / Multi Factor Authentication (MFA) requires anyone entering to have two keys to enter which makes it harder for bad actors. The two keys are:

1. Their password
2. A 6 digit token that changes every 30 seconds

Without both keys, they won't be able to get in.

Why Not

Many folks might find the experience cumbersome or annoying. And, access to your system is dependent on the accompanying authenticator app that generates the code.

How It Works

On Linux, PAM is responsible for authentication. There are four tasks to PAM that you can read about at https://en.wikipedia.org/wiki/Linux_PAM. This section talks about the authentication task.

When you log into a server, be it directly from the console or via SSH, the door you came through will send the request to the authentication task of PAM and PAM will ask for and verify your password. You can customize the rules each doors use. For example, you could have one set of rules when logging in directly from the console and another set of rules for when logging in via SSH.

This section will alter the authentication rules for when logging in via SSH to require both a password and a 6 digit code.

We will use Google's libpam-google-authenticator PAM module to create and verify a TOTP key. https://fastmail.blog/2016/07/22/how-totp-authenticator-apps-work/ and https://jemurai.com/2018/10/11/how-it-works-totp-based-mfa/ have very good writeups of how TOTP works.

What we will do is tell the server's SSH PAM configuration to ask the user for their password and then their numeric token. PAM will then verify the user's password and, if it is correct, then it will route the authentication request to libpam-google-authenticator which will ask for and verify your 6 digit token. If, and only if, everything is good will the authentication succeed and user be allowed to log in.

Goals

• 2FA/MFA enabled for all SSH connections

Notes

• Before you do this, you should have an idea of how 2FA/MFA works and you'll need an authenticator app on your phone to continue.
• We'll use google-authenticator-libpam.
• With the below configuration, a user will only need to enter their 2FA/MFA code if they are logging on with their password but not if they are using SSH public/private keys. Check the documentation on how to change this behavior to suite your requirements.

References

• https://github.com/google/google-authenticator-libpam
• https://en.wikipedia.org/wiki/Linux_PAM
• https://en.wikipedia.org/wiki/Time-based_One-time_Password_algorithm
• https://fastmail.blog/2016/07/22/how-totp-authenticator-apps-work/
• https://jemurai.com/2018/10/11/how-it-works-totp-based-mfa/

Steps

Install it libpam-google-authenticator.

On Debian based systems:

sudo apt install libpam-google-authenticator

Make sure you're logged in as the ID you want to enable 2FA/MFA for and execute google-authenticator to create the necessary token data:

google-authenticator

Do you want authentication tokens to be time-based (y/n) y
https://www.google.com/chart?chs=200x200&chld=M|0&cht=qr&chl=otpauth://totp/user@host%3Fsecret%3DR4ZWX34FQKZROVX7AGLJ64684Y%26issuer%3Dhost

...

Your new secret key is: R3NVX3FFQKZROVX7AGLJUGGESY
Your verification code is 751419
Your emergency scratch codes are:
  12345678
  90123456
  78901234
  56789012
  34567890

Do you want me to update your "/home/user/.google_authenticator" file (y/n) y

Do you want to disallow multiple uses of the same authentication
token? This restricts you to one login about every 30s, but it increases
your chances to notice or even prevent man-in-the-middle attacks (y/n) Do you want to disallow multiple uses of the same authentication
token? This restricts you to one login about every 30s, but it increases
your chances to notice or even prevent man-in-the-middle attacks (y/n) y

By default, tokens are good for 30 seconds. In order to compensate for
possible time-skew between the client and the server, we allow an extra
token before and after the current time. If you experience problems with
poor time synchronization, you can increase the window from its default
size of +-1min (window size of 3) to about +-4min (window size of
17 acceptable tokens).
Do you want to do so? (y/n) y

If the computer that you are logging into isn't hardened against brute-force
login attempts, you can enable rate-limiting for the authentication module.
By default, this limits attackers to no more than 3 login attempts every 30s.
Do you want to enable rate-limiting (y/n) y

Notice this is not run as root.

Select default option (y in most cases) for all the questions it asks and remember to save the emergency scratch codes.

Make a backup of PAM's SSH configuration file /etc/pam.d/sshd:

sudo cp --archive /etc/pam.d/sshd /etc/pam.d/sshd-COPY-$(date +"%Y%m%d%H%M%S")

Now we need to enable it as an authentication method for SSH by adding this line to /etc/pam.d/sshd:

auth       required     pam_google_authenticator.so nullok

Note: Check here for what nullok means.

For the lazy:

echo -e "\nauth       required     pam_google_authenticator.so nullok         # added by $(whoami) on $(date +"%Y-%m-%d @ %H:%M:%S")" | sudo tee -a /etc/pam.d/sshd

Tell SSH to leverage it by adding or editing this line in /etc/ssh/sshd_config:

ChallengeResponseAuthentication yes

For the lazy:

sudo sed -i -r -e "s/^(challengeresponseauthentication .*)$/# \1         # commented by $(whoami) on $(date +"%Y-%m-%d @ %H:%M:%S")/I" /etc/ssh/sshd_config
echo -e "\nChallengeResponseAuthentication yes         # added by $(whoami) on $(date +"%Y-%m-%d @ %H:%M:%S")" | sudo tee -a /etc/ssh/sshd_config

Restart ssh:

sudo service sshd restart

The Basics

Limit Who Can Use sudo

Why

sudo lets accounts run commands as other accounts, including root. We want to make sure that only the accounts we want can use sudo.

Goals

• sudo privileges limited to those who are in a group we specify

Notes

• Your installation may have already done this, or may already have a special group intended for this purpose so check first.
  • Debian creates the sudo group
  • RedHat creates the wheel group
• See https://github.com/imthenachoman/How-To-Secure-A-Linux-Server/issues/39 for a note on some distributions making it so sudo does not require a password. Thanks to sbrl for sharing.

Steps

Create a group:

sudo groupadd sudousers

Add account(s) to the group:

sudo usermod -a -G sudousers user1
sudo usermod -a -G sudousers user2
sudo usermod -a -G sudousers  ...

You'll need to do this for every account on your server that needs sudo privileges.

Make a backup of the sudo's configuration file /etc/sudoers:

sudo cp --archive /etc/sudoers /etc/sudoers-COPY-$(date +"%Y%m%d%H%M%S")

Edit sudo's configuration file /etc/sudoers:

sudo visudo

Tell sudo to only allow users in the sudousers group to use sudo by adding this line if it is not already there:

%sudousers   ALL=(ALL:ALL) ALL

Limit Who Can Use su

Why

su also lets accounts run commands as other accounts, including root. We want to make sure that only the accounts we want can use su.

Goals

• su privileges limited to those who are in a group we specify

References

• Thanks to olavim for sharing this idea

Steps

Create a group:

sudo groupadd suusers

Add account(s) to the group:

sudo usermod -a -G suusers user1
sudo usermod -a -G suusers user2
sudo usermod -a -G suusers  ...

You'll need to do this for every account on your server that needs sudo privileges.

Make it so only users in this group can execute /bin/su:

sudo dpkg-statoverride --update --add root suusers 4750 /bin/su

Run applications in a sandbox with FireJail

Why

It's absolutely better, for many applications, to run in a sandbox.

Browsers (even more the Closed Source ones) and eMail Clients are highly suggested.

Goals

• confine applications in a jail (few safe directories) and block access to the rest of the system

References

• Thanks to FireJail

Steps

Install the software:

sudo apt install firejail firejail-profiles

Note: for Debian 10 Stable, official Backport is suggested:

sudo apt install -t buster-backports firejail firejail-profiles

Allow an application (installed in /usr/bin or /bin) to run only in a sandbox (see few examples below here):

sudo ln -s /usr/bin/firejail /usr/local/bin/google-chrome-stable
sudo ln -s /usr/bin/firejail /usr/local/bin/firefox
sudo ln -s /usr/bin/firejail /usr/local/bin/chromium
sudo ln -s /usr/bin/firejail /usr/local/bin/evolution
sudo ln -s /usr/bin/firejail /usr/local/bin/thunderbird

Run the application as usual (via terminal or launcher) and check if is running in a jail:

firejail --list

Allow a sandboxed app to run again as it was before (example: firefox)

sudo rm /usr/local/bin/firefox

NTP Client

Why

Many security protocols leverage the time. If your system time is incorrect, it could have negative impacts to your server. An NTP client can solve that problem by keeping your system time in-sync with global NTP servers

How It Works

NTP stands for Network Time Protocol. In the context of this guide, an NTP client on the server is used to update the server time with the official time pulled from official servers. Check https://www.pool.ntp.org/en/ for all of the public NTP servers.

Goals

• NTP client installed and keeping server time in-sync

References

• https://cloudpro.zone/index.php/2018/01/27/debian-9-3-server-setup-guide-part-4/
• https://en.wikipedia.org/wiki/Network_Time_Protocol
• https://www.pool.ntp.org/en/
• https://serverfault.com/questions/957302/securing-hardening-ntp-client-on-linux-servers-config-file/957450#957450
• https://tf.nist.gov/tf-cgi/servers.cgi

Steps

Install ntp.

On Debian based systems:

sudo apt install ntp

Make a backup of the NTP client's configuration file /etc/ntp.conf:

sudo cp --archive /etc/ntp.conf /etc/ntp.conf-COPY-$(date +"%Y%m%d%H%M%S")

The default configuration, at least on Debian, is already pretty secure. The only thing we'll want to make sure is we're the pool directive and not any server directives. The pool directive allows the NTP client to stop using a server if it is unresponsive or serving bad time. Do this by commenting out all server directives and adding the below to /etc/ntp.conf.

pool pool.ntp.org iburst

For the lazy:

sudo sed -i -r -e "s/^((server|pool).*)/# \1         # commented by $(whoami) on $(date +"%Y-%m-%d @ %H:%M:%S")/" /etc/ntp.conf
echo -e "\npool pool.ntp.org iburst         # added by $(whoami) on $(date +"%Y-%m-%d @ %H:%M:%S")" | sudo tee -a /etc/ntp.conf

Example /etc/ntp.conf:

driftfile /var/lib/ntp/ntp.drift
statistics loopstats peerstats clockstats
filegen loopstats file loopstats type day enable
filegen peerstats file peerstats type day enable
filegen clockstats file clockstats type day enable
restrict -4 default kod notrap nomodify nopeer noquery limited
restrict -6 default kod notrap nomodify nopeer noquery limited
restrict 127.0.0.1
restrict ::1
restrict source notrap nomodify noquery
pool pool.ntp.org iburst         # added by user on 2019-03-09 @ 10:23:35

Restart ntp:

sudo service ntp restart

Check the status of the ntp service:

sudo systemctl status ntp

● ntp.service - LSB: Start NTP daemon
   Loaded: loaded (/etc/init.d/ntp; generated; vendor preset: enabled)
   Active: active (running) since Sat 2019-03-09 15:19:46 EST; 4s ago
     Docs: man:systemd-sysv-generator(8)
  Process: 1016 ExecStop=/etc/init.d/ntp stop (code=exited, status=0/SUCCESS)
  Process: 1028 ExecStart=/etc/init.d/ntp start (code=exited, status=0/SUCCESS)
    Tasks: 2 (limit: 4915)
   CGroup: /system.slice/ntp.service
           └─1038 /usr/sbin/ntpd -p /var/run/ntpd.pid -g -u 108:113

Mar 09 15:19:46 host ntpd[1038]: Listen and drop on 0 v6wildcard [::]:123
Mar 09 15:19:46 host ntpd[1038]: Listen and drop on 1 v4wildcard 0.0.0.0:123
Mar 09 15:19:46 host ntpd[1038]: Listen normally on 2 lo 127.0.0.1:123
Mar 09 15:19:46 host ntpd[1038]: Listen normally on 3 enp0s3 10.10.20.96:123
Mar 09 15:19:46 host ntpd[1038]: Listen normally on 4 lo [::1]:123
Mar 09 15:19:46 host ntpd[1038]: Listen normally on 5 enp0s3 [fe80::a00:27ff:feb6:ed8e%2]:123
Mar 09 15:19:46 host ntpd[1038]: Listening on routing socket on fd #22 for interface updates
Mar 09 15:19:47 host ntpd[1038]: Soliciting pool server 108.61.56.35
Mar 09 15:19:48 host ntpd[1038]: Soliciting pool server 69.89.207.199
Mar 09 15:19:49 host ntpd[1038]: Soliciting pool server 45.79.111.114

Check ntp's status:

sudo ntpq -p

     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
 pool.ntp.org    .POOL.          16 p    -   64    0    0.000    0.000   0.000
*lithium.constan 198.30.92.2      2 u    -   64    1   19.900    4.894   3.951
 ntp2.wiktel.com 212.215.1.157    2 u    2   64    1   48.061   -0.431   0.104

Securing /proc

Why

To quote https://linux-audit.com/linux-system-hardening-adding-hidepid-to-proc/:

When looking in /proc you will discover a lot of files and directories. Many of them are just numbers, which represent the information about a particular process ID (PID). By default, Linux systems are deployed to allow all local users to see this all information. This includes process information from other users. This could include sensitive details that you may not want to share with other users. By applying some filesystem configuration tweaks, we can change this behavior and improve the security of the system.

Note: This may break on some systemd systems. Please see https://github.com/imthenachoman/How-To-Secure-A-Linux-Server/issues/37 for more information. Thanks to nlgranger for sharing.

Goals

• /proc mounted with hidepid=2 so users can only see information about their processes

References

• https://linux-audit.com/linux-system-hardening-adding-hidepid-to-proc/
• https://likegeeks.com/secure-linux-server-hardening-best-practices/#Hardening-proc-Directory
• https://www.cyberciti.biz/faq/linux-hide-processes-from-other-users/

Steps

Make a backup of /etc/fstab:

sudo cp --archive /etc/fstab /etc/fstab-COPY-$(date +"%Y%m%d%H%M%S")

Add this line to /etc/fstab to have /proc mounted with hidepid=2:

proc     /proc     proc     defaults,hidepid=2     0     0

For the lazy:

echo -e "\nproc     /proc     proc     defaults,hidepid=2     0     0         # added by $(whoami) on $(date +"%Y-%m-%d @ %H:%M:%S")" | sudo tee -a /etc/fstab

Reboot the system:

sudo reboot now

Note: Alternatively, you can remount /proc without rebooting with sudo mount -o remount,hidepid=2 /proc

Force Accounts To Use Secure Passwords

Why

By default, accounts can use any password they want, including bad ones. pwquality/pam_pwquality addresses this security gap by providing "a way to configure the default password quality requirements for the system passwords" and checking "its strength against a system dictionary and a set of rules for identifying poor choices."

How It Works

On Linux, PAM is responsible for authentication. There are four tasks to PAM that you can read about at https://en.wikipedia.org/wiki/Linux_PAM. This section talks about the password task.

When there is a need to set or change an account password, the password task of PAM handles the request. In this section we will tell PAM's password task to pass the requested new password to libpam-pwquality to make sure it meets our requirements. If the requirements are met it is used/set; if it does not meet the requirements it errors and lets the user know.

Goals

• enforced strong passwords

Steps

Install libpam-pwquality.

On Debian based systems:

sudo apt install libpam-pwquality

Make a backup of PAM's password configuration file /etc/pam.d/common-password:

sudo cp --archive /etc/pam.d/common-password /etc/pam.d/common-password-COPY-$(date +"%Y%m%d%H%M%S")

Tell PAM to use libpam-pwquality to enforce strong passwords by editing the file /etc/pam.d/common-password and change the line that starts like this:

password        requisite                       pam_pwquality.so

to this:

password        requisite                       pam_pwquality.so retry=3 minlen=10 difok=3 ucredit=-1 lcredit=-1 dcredit=-1 ocredit=-1 maxrepeat=3 gecoschec

The above options are:

• retry=3 = prompt user 3 times before returning with error.
• minlen=10 = the minimum length of the password, factoring in any credits (or debits) from these:
  • dcredit=-1 = must have at least one digit
  • ucredit=-1 = must have at least one upper case letter
  • lcredit=-1 = must have at least one lower case letter
  • ocredit=-1 = must have at least one non-alphanumeric character
• difok=3 = at least 3 characters from the new password cannot have been in the old password
• maxrepeat=3 = allow a maximum of 3 repeated characters
• gecoschec = do not allow passwords with the account's name

sudo sed -i -r -e "s/^(password\s+requisite\s+pam_pwquality.so)(.*)$/# \1\2         # commented by $(whoami) on $(date +"%Y-%m-%d @ %H:%M:%S")\n\1 retry=3 minlen=10 difok=3 ucredit=-1 lcredit=-1 dcredit=-1 ocredit=-1 maxrepeat=3 gecoschec         # added by $(whoami) on $(date +"%Y-%m-%d @ %H:%M:%S")/" /etc/pam.d/common-password

Automatic Security Updates and Alerts

Why

It is important to keep a server updated with the latest critical security patches and updates. Otherwise you're at risk of known security vulnerabilities that bad-actors could use to gain unauthorized access to your server.

Unless you plan on checking your server every day, you'll want a way to automatically update the system and/or get emails about available updates.

You don't want to do all updates because with every update there is a risk of something breaking. It is important to do the critical updates but everything else can wait until you have time to do it manually.

Why Not

Automatic and unattended updates may break your system and you may not be near your server to fix it. This would be especially problematic if it broke your SSH access.

Notes

• Each distribution manages packages and updates differently. So far I only have steps for Debian based systems.
• Your server will need a way to send e-mails for this to work

Goals

• Automatic, unattended, updates of critical security patches
• Automatic emails of remaining pending updates

Debian Based Systems

How It Works

On Debian based systems you can use:

• unattended-upgrades to automatically do system updates you want (i.e. critical security updates)
• apt-listchanges to get details about package changes before they are installed/upgraded
• apticron to get emails for pending package updates

We will use unattended-upgrades to apply critical security patches. We can also apply stable updates since they've already been thoroughly tested by the Debian community.

References

• https://wiki.debian.org/UnattendedUpgrades
• https://debian-handbook.info/browse/stable/sect.regular-upgrades.html
• https://blog.sleeplessbeastie.eu/2015/01/02/how-to-perform-unattended-upgrades/
• https://www.vultr.com/docs/how-to-set-up-unattended-upgrades-on-debian-9-stretch
• https://github.com/mvo5/unattended-upgrades
• https://wiki.debian.org/UnattendedUpgrades#apt-listchanges
• https://www.cyberciti.biz/faq/apt-get-apticron-send-email-upgrades-available/
• https://www.unixmen.com/how-to-get-email-notifications-for-new-updates-on-debianubuntu/
• /etc/apt/apt.conf.d/50unattended-upgrades

Steps

Install unattended-upgrades, apt-listchanges, and apticron:

sudo apt install unattended-upgrades apt-listchanges apticron

Now we need to configure unattended-upgrades to automatically apply the updates. This is typically done by editing the files /etc/apt/apt.conf.d/20auto-upgrades and /etc/apt/apt.conf.d/50unattended-upgrades that were created by the packages. However, because these file may get overwritten with a future update, we'll create a new file instead. Create the file /etc/apt/apt.conf.d/51myunattended-upgrades and add this:

// Enable the update/upgrade script (0=disable)
APT::Periodic::Enable "1";

// Do "apt-get update" automatically every n-days (0=disable)
APT::Periodic::Update-Package-Lists "1";

// Do "apt-get upgrade --download-only" every n-days (0=disable)
APT::Periodic::Download-Upgradeable-Packages "1";

// Do "apt-get autoclean" every n-days (0=disable)
APT::Periodic::AutocleanInterval "7";

// Send report mail to root
//     0:  no report             (or null string)
//     1:  progress report       (actually any string)
//     2:  + command outputs     (remove -qq, remove 2>/dev/null, add -d)
//     3:  + trace on    APT::Periodic::Verbose "2";
APT::Periodic::Unattended-Upgrade "1";

// Automatically upgrade packages from these
Unattended-Upgrade::Origins-Pattern {
      "o=Debian,a=stable";
      "o=Debian,a=stable-updates";
      "origin=Debian,codename=${distro_codename},label=Debian-Security";
};

// You can specify your own packages to NOT automatically upgrade here
Unattended-Upgrade::Package-Blacklist {
};

// Run dpkg --force-confold --configure -a if a unclean dpkg state is detected to true to ensure that updates get installed even when the system got interrupted during a previous run
Unattended-Upgrade::AutoFixInterruptedDpkg "true";

//Perform the upgrade when the machine is running because we wont be shutting our server down often
Unattended-Upgrade::InstallOnShutdown "false";

// Send an email to this address with information about the packages upgraded.
Unattended-Upgrade::Mail "root";

// Always send an e-mail
Unattended-Upgrade::MailOnlyOnError "false";

// Remove all unused dependencies after the upgrade has finished
Unattended-Upgrade::Remove-Unused-Dependencies "true";

// Remove any new unused dependencies after the upgrade has finished
Unattended-Upgrade::Remove-New-Unused-Dependencies "true";

// Automatically reboot WITHOUT CONFIRMATION if the file /var/run/reboot-required is found after the upgrade.
Unattended-Upgrade::Automatic-Reboot "true";

// Automatically reboot even if users are logged in.
Unattended-Upgrade::Automatic-Reboot-WithUsers "true";

Notes:

• Check /usr/lib/apt/apt.systemd.daily for details on the APT::Periodic options
• Check https://github.com/mvo5/unattended-upgrades for details on the Unattended-Upgrade options

Run a dry-run of unattended-upgrades to make sure your configuration file is okay:

sudo unattended-upgrade -d --dry-run

If everything is okay, you can let it run whenever it's scheduled to or force a run with unattended-upgrade -d.

Configure apt-listchanges to your liking:

sudo dpkg-reconfigure apt-listchanges

For apticron, the default settings are good enough but you can check them in /etc/apticron/apticron.conf if you want to change them. For example, my configuration looks like this:

EMAIL="root"
NOTIFY_NO_UPDATES="1"

More Secure Random Entropy Pool (WIP)

Why

WIP

How It Works

WIP

Goals

WIP

References

• Thanks to branneman for this idea as submitted in issue #33.
• https://hackaday.com/2017/11/02/what-is-entropy-and-how-do-i-get-more-of-it/
• https://www.2uo.de/myths-about-urandom
• https://www.gnu.org/software/hurd/user/tlecarrour/rng-tools.html
• https://wiki.archlinux.org/index.php/Rng-tools
• https://www.howtoforge.com/helping-the-random-number-generator-to-gain-enough-entropy-with-rng-tools-debian-lenny
• https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/security_guide/sect-security_guide-encryption-using_the_random_number_generator

Steps

Install rng-tools.

On Debian based systems:

sudo apt-get install rng-tools

Now we need to set the hardware device used to generate random numbers by adding this to /etc/default/rng-tools:

HRNGDEVICE=/dev/urandom

For the lazy:

echo "HRNGDEVICE=/dev/urandom" | sudo tee -a /etc/default/rng-tools

Restart the service:

sudo systemctl stop rng-tools.service
sudo systemctl start rng-tools.service

Test randomness:

• https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/security_guide/sect-security_guide-encryption-using_the_random_number_generator
• https://wiki.archlinux.org/index.php/Rng-tools

The Network

Firewall With UFW (Uncomplicated Firewall)

Why

Call me paranoid, and you don't have to agree, but I want to deny all traffic in and out of my server except what I explicitly allow. Why would my server be sending traffic out that I don't know about? And why would external traffic be trying to access my server if I don't know who or what it is? When it comes to good security, my opinion is to reject/deny by default, and allow by exception.

Of course, if you disagree, that is totally fine and can configure UFW to suit your needs.

Either way, ensuring that only traffic we explicitly allow is the job of a firewall.

How It Works

The Linux kernel provides capabilities to monitor and control network traffic. These capabilities are exposed to the end-user through firewall utilities. On Linux, the most common firewall is iptables. However, iptables is rather complicated and confusing (IMHO). This is where UFW comes in. Think of UFW as a front-end to iptables. It simplifies the process of managing the iptables rules that tell the Linux kernel what to do with network traffic.

UFW works by letting you configure rules that:

• allow or deny
• input or output traffic
• to or from ports

You can create rules by explicitly specifying the ports or with application configurations that specify the ports.

Goals

• all network traffic, input and output, blocked except those we explicitly allow

Notes

• As you install other programs, you'll need to enable the necessary ports/applications.

References

• https://launchpad.net/ufw

Steps

Install ufw.

On Debian based systems:

sudo apt install ufw

Deny all outgoing traffic:

sudo ufw default deny outgoing comment 'deny all outgoing traffic'

Default outgoing policy changed to 'deny'
(be sure to update your rules accordingly)

If you are not as paranoid as me, and don't want to deny all outgoing traffic, you can allow it instead:

sudo ufw default allow outgoing comment 'allow all outgoing traffic'

Deny all incoming traffic:

sudo ufw default deny incoming comment 'deny all incoming traffic'

Obviously we want SSH connections in:

sudo ufw limit in ssh comment 'allow SSH connections in'

Rules updated
Rules updated (v6)

Allow additional traffic as per your needs. Some common use-cases:

# allow traffic out on port 53 -- DNS
sudo ufw allow out 53 comment 'allow DNS calls out'

# allow traffic out on port 123 -- NTP
sudo ufw allow out 123 comment 'allow NTP out'

# allow traffic out for HTTP, HTTPS, or FTP
# apt might needs these depending on which sources you're using
sudo ufw allow out http comment 'allow HTTP traffic out'
sudo ufw allow out https comment 'allow HTTPS traffic out'
sudo ufw allow out ftp comment 'allow FTP traffic out'

# allow whois
sudo ufw allow out whois comment 'allow whois'

# allow traffic out on port 68 -- the DHCP client
# you only need this if you're using DHCP
sudo ufw allow out 67 comment 'allow the DHCP client to update'
sudo ufw allow out 68 comment 'allow the DHCP client to update'

Start ufw:

sudo ufw enable

Command may disrupt existing ssh connections. Proceed with operation (y|n)? y
Firewall is active and enabled on system startup

If you want to see a status:

sudo ufw status

Status: active

To                         Action      From
--                         ------      ----
22/tcp                     LIMIT       Anywhere                   # allow SSH connections in
22/tcp (v6)                LIMIT       Anywhere (v6)              # allow SSH connections in

53                         ALLOW OUT   Anywhere                   # allow DNS calls out
123                        ALLOW OUT   Anywhere                   # allow NTP out
80/tcp                     ALLOW OUT   Anywhere                   # allow HTTP traffic out
443/tcp                    ALLOW OUT   Anywhere                   # allow HTTPS traffic out
21/tcp                     ALLOW OUT   Anywhere                   # allow FTP traffic out
Mail submission            ALLOW OUT   Anywhere                   # allow mail out
43/tcp                     ALLOW OUT   Anywhere                   # allow whois
53 (v6)                    ALLOW OUT   Anywhere (v6)              # allow DNS calls out
123 (v6)                   ALLOW OUT   Anywhere (v6)              # allow NTP out
80/tcp (v6)                ALLOW OUT   Anywhere (v6)              # allow HTTP traffic out
443/tcp (v6)               ALLOW OUT   Anywhere (v6)              # allow HTTPS traffic out
21/tcp (v6)                ALLOW OUT   Anywhere (v6)              # allow FTP traffic out
Mail submission (v6)       ALLOW OUT   Anywhere (v6)              # allow mail out
43/tcp (v6)                ALLOW OUT   Anywhere (v6)              # allow whois

or

sudo ufw status verbose

Status: active
Logging: on (low)
Default: deny (incoming), deny (outgoing), disabled (routed)
New profiles: skip

To                         Action      From
--                         ------      ----
22/tcp                     LIMIT IN    Anywhere                   # allow SSH connections in
22/tcp (v6)                LIMIT IN    Anywhere (v6)              # allow SSH connections in

53                         ALLOW OUT   Anywhere                   # allow DNS calls out
123                        ALLOW OUT   Anywhere                   # allow NTP out
80/tcp                     ALLOW OUT   Anywhere                   # allow HTTP traffic out
443/tcp                    ALLOW OUT   Anywhere                   # allow HTTPS traffic out
21/tcp                     ALLOW OUT   Anywhere                   # allow FTP traffic out
587/tcp (Mail submission)  ALLOW OUT   Anywhere                   # allow mail out
43/tcp                     ALLOW OUT   Anywhere                   # allow whois
53 (v6)                    ALLOW OUT   Anywhere (v6)              # allow DNS calls out
123 (v6)                   ALLOW OUT   Anywhere (v6)              # allow NTP out
80/tcp (v6)                ALLOW OUT   Anywhere (v6)              # allow HTTP traffic out
443/tcp (v6)               ALLOW OUT   Anywhere (v6)              # allow HTTPS traffic out
21/tcp (v6)                ALLOW OUT   Anywhere (v6)              # allow FTP traffic out
587/tcp (Mail submission (v6)) ALLOW OUT   Anywhere (v6)              # allow mail out
43/tcp (v6)                ALLOW OUT   Anywhere (v6)              # allow whois

Default Applications

ufw ships with some default applications. You can see them with:

sudo ufw app list

Available applications:
  AIM
  Bonjour
  CIFS
  DNS
  Deluge
  IMAP
  IMAPS
  IPP
  KTorrent
  Kerberos Admin
  Kerberos Full
  Kerberos KDC
  Kerberos Password
  LDAP
  LDAPS
  LPD
  MSN
  MSN SSL
  Mail submission
  NFS
  OpenSSH
  POP3
  POP3S
  PeopleNearby
  SMTP
  SSH
  Socks
  Telnet
  Transmission
  Transparent Proxy
  VNC
  WWW
  WWW Cache
  WWW Full
  WWW Secure
  XMPP
  Yahoo
  qBittorrent
  svnserve

To get details about the app, like which ports it includes, type:

sudo ufw app info [app name]

sudo ufw app info DNS

Profile: DNS
Title: Internet Domain Name Server
Description: Internet Domain Name Server

Port:
  53

Custom Application

If you don't want to create rules by explicitly providing the port number(s), you can create your own application configurations. To do this, create a file in /etc/ufw/applications.d.

For example, here is what you would use for Plex:

cat /etc/ufw/applications.d/plexmediaserver

[PlexMediaServer]
title=Plex Media Server
description=This opens up PlexMediaServer for http (32400), upnp, and autodiscovery.
ports=32469/tcp|32413/udp|1900/udp|32400/tcp|32412/udp|32410/udp|32414/udp|32400/udp

Then you can enable it like any other app:

sudo ufw allow plexmediaserver

iptables Intrusion Detection And Prevention with PSAD

Why

Even if you have a firewall to guard your doors, it is possible to try brute-forcing your way in any of the guarded doors. We want to monitor all network activity to detect potential intrusion attempts, such has repeated attempts to get in, and block them.

How It Works

I can't explain it any better than user FINESEC from https://serverfault.com/ did at: https://serverfault.com/a/447604/289829.

Fail2BAN scans log files of various applications such as apache, ssh or ftp and automatically bans IPs that show the malicious signs such as automated login attempts. PSAD on the other hand scans iptables and ip6tables log messages (typically /var/log/messages) to detect and optionally block scans and other types of suspect traffic such as DDoS or OS fingerprinting attempts. It's ok to use both programs at the same time because they operate on different level.

And, since we're already using UFW so we'll follow the awesome instructions by netson at https://gist.github.com/netson/c45b2dc4e835761fbccc to make PSAD work with UFW.

References

• http://www.cipherdyne.org/psad/
• http://www.cipherdyne.org/psad/docs/config.html
• https://www.thefanclub.co.za/how-to/how-install-psad-intrusion-detection-ubuntu-1204-lts-server
• https://serverfault.com/a/447604/289829
• https://serverfault.com/a/770424/289829
• https://gist.github.com/netson/c45b2dc4e835761fbccc
• Thanks to sysadt for catching the issue (#61) with psadwatchd.

Steps

Install psad.

On Debian based systems:

sudo apt install psad

Make a backup of psad's configuration file /etc/psad/psad.conf:

sudo cp --archive /etc/psad/psad.conf /etc/psad/psad.conf-COPY-$(date +"%Y%m%d%H%M%S")

Review and update configuration options in /etc/psad/psad.conf. Pay special attention to these:

Setting	Set To
EMAIL_ADDRESSES	your email address(s)
HOSTNAME	your server's hostname
ENABLE_PSADWATCHD	ENABLE_PSADWATCHD Y;
ENABLE_AUTO_IDS	ENABLE_AUTO_IDS Y;
ENABLE_AUTO_IDS_EMAILS	ENABLE_AUTO_IDS_EMAILS Y;
EXPECT_TCP_OPTIONS	EXPECT_TCP_OPTIONS Y;

Check the configuration file psad's documentation at http://www.cipherdyne.org/psad/docs/config.html for more details.

Now we need to make some changes to ufw so it works with psad by telling ufw to log all traffic so psad can analyze it. Do this by editing two files and adding these lines at the end but before the COMMIT line.

Make backups:

sudo cp --archive /etc/ufw/before.rules /etc/ufw/before.rules-COPY-$(date +"%Y%m%d%H%M%S")
sudo cp --archive /etc/ufw/before6.rules /etc/ufw/before6.rules-COPY-$(date +"%Y%m%d%H%M%S")

Edit the files:

• /etc/ufw/before.rules
• /etc/ufw/before6.rules

Now we need to reload/restart ufw and psad for the changes to take effect:

sudo ufw reload

sudo psad -R
sudo psad --sig-update
sudo psad -H

Analyze iptables rules for errors:

sudo psad --fw-analyze

[+] Parsing INPUT chain rules.
[+] Parsing INPUT chain rules.
[+] Firewall config looks good.
[+] Completed check of firewall ruleset.
[+] Results in /var/log/psad/fw_check
[+] Exiting.

Note: If there were any issues you will get an e-mail with the error.

Check the status of psad:

sudo psad --Status

[-] psad: pid file /var/run/psad/psadwatchd.pid does not exist for psadwatchd on vm
[+] psad_fw_read (pid: 3444)  %CPU: 0.0  %MEM: 2.2
    Running since: Sat Feb 16 01:03:09 2019

[+] psad (pid: 3435)  %CPU: 0.2  %MEM: 2.7
    Running since: Sat Feb 16 01:03:09 2019
    Command line arguments: [none specified]
    Alert email address(es): root@localhost

[+] Version: psad v2.4.3

[+] Top 50 signature matches:
        [NONE]

[+] Top 25 attackers:
        [NONE]

[+] Top 20 scanned ports:
        [NONE]

[+] iptables log prefix counters:
        [NONE]

    Total protocol packet counters:

[+] IP Status Detail:
        [NONE]

    Total scan sources: 0
    Total scan destinations: 0

[+] These results are available in: /var/log/psad/status.out

Application Intrusion Detection And Prevention With Fail2Ban

Why

UFW tells your server what doors to board up so nobody can see them, and what doors to allow authorized users through. PSAD monitors network activity to detect and prevent potential intrusions -- repeated attempts to get in.

But what about the applications/services your server is running, like SSH and Apache, where your firewall is configured to allow access in. Even though access may be allowed that doesn't mean all access attempts are valid and harmless. What if someone tries to brute-force their way in to a web-app you're running on your server? This is where Fail2ban comes in.

How It Works

Fail2ban monitors the logs of your applications (like SSH and Apache) to detect and prevent potential intrusions. It will monitor network traffic/logs and prevent intrusions by blocking suspicious activity (e.g. multiple successive failed connections in a short time-span).

Goals

• network monitoring for suspicious activity with automatic banning of offending IPs

Notes

• As of right now, the only thing running on this server is SSH so we'll want Fail2ban to monitor SSH and ban as necessary.
• As you install other programs, you'll need to create/configure the appropriate jails and enable them.

References

• https://www.fail2ban.org/
• https://blog.vigilcode.com/2011/05/ufw-with-fail2ban-quick-secure-setup-part-ii/
• https://dodwell.us/security/ufw-fail2ban-portscan.html
• https://www.howtoforge.com/community/threads/fail2ban-and-ufw-on-debian.77261/

Steps

Install fail2ban.

On Debian based systems:

sudo apt install fail2ban

We don't want to edit /etc/fail2ban/fail2ban.conf or /etc/fail2ban/jail.conf because a future update may overwrite those so we'll create a local copy instead. Create the file /etc/fail2ban/jail.local and add this to it after replacing [LAN SEGMENT] and [your email] with the appropriate values:

[DEFAULT]
# the IP address range we want to ignore
ignoreip = 127.0.0.1/8 [LAN SEGMENT]

# who to send e-mail to
destemail = [your e-mail]

# who is the email from
sender = [your e-mail]

# since we're using exim4 to send emails
mta = mail

# get email alerts
action = %(action_mwl)s

Note: Your server will need to be able to send e-mails so Fail2ban can let you know of suspicious activity and when it banned an IP.

We need to create a jail for SSH that tells fail2ban to look at SSH logs and use ufw to ban/unban IPs as needed. Create a jail for SSH by creating the file /etc/fail2ban/jail.d/ssh.local and adding this to it:

[sshd]
enabled = true
banaction = ufw
port = ssh
filter = sshd
logpath = %(sshd_log)s
maxretry = 5

For the lazy:

cat << EOF | sudo tee /etc/fail2ban/jail.d/ssh.local
[sshd]
enabled = true
banaction = ufw
port = ssh
filter = sshd
logpath = %(sshd_log)s
maxretry = 5
EOF

In the above we tell fail2ban to use the ufw as the banaction. Fail2ban ships with an action configuration file for ufw. You can see it in /etc/fail2ban/action.d/ufw.conf

Enable fail2ban:

sudo fail2ban-client start
sudo fail2ban-client reload
sudo fail2ban-client add sshd # This may fail on some systems if the sshd jail was added by default

To check the status:

sudo fail2ban-client status

Status
|- Number of jail:      1
`- Jail list:   sshd

sudo fail2ban-client status sshd

Status for the jail: sshd
|- Filter
|  |- Currently failed: 0
|  |- Total failed:     0
|  `- File list:        /var/log/auth.log
`- Actions
   |- Currently banned: 0
   |- Total banned:     0
   `- Banned IP list:

Custom Jails

I have not needed to create a custom jail yet. Once I do, and I figure out how, I will update this guide. Or, if you know how please help contribute.

Unban an IP

To unban an IP use this command:

fail2ban-client set [jail] unbanip [IP]

[jail] is the name of the jail that has the banned IP and [IP] is the IP address you want to unban. For example, to unaban 192.168.1.100 from SSH you would do:

fail2ban-client set sshd unbanip 192.168.1.100

The Auditing

File/Folder Integrity Monitoring With AIDE (WIP)

Why

WIP

How It Works

WIP

Goals

WIP

References

• https://aide.github.io/
• https://www.hiroom2.com/2017/06/09/debian-8-file-integrity-check-with-aide/
• https://blog.rapid7.com/2017/06/30/how-to-install-and-configure-aide-on-ubuntu-linux/
• https://www.stephenrlang.com/2016/03/using-aide-for-file-integrity-monitoring-fim-on-ubuntu/
• https://www.howtoforge.com/how-to-configure-the-aide-advanced-intrusion-detection-environment-file-integrity-scanner-for-your-website
• https://www.tecmint.com/check-integrity-of-file-and-directory-using-aide-in-linux/
• https://www.cyberciti.biz/faq/debian-ubuntu-linux-software-integrity-checking-with-aide/

Steps

Install AIDE.

On Debian based systems:

sudo apt install aide

Make a backup of AIDE's defaults file:

sudo cp -p /etc/default/aide /etc/default/aide-COPY-$(date +"%Y%m%d%H%M%S")

Go through /etc/default/aide and set AIDE's defaults per your requirements. If you want AIDE to run daily and e-mail you, be sure to set CRON_DAILY_RUN to yes.

Make a backup of AIDE's configuration files:

sudo cp -pr /etc/aide /etc/aide-COPY-$(date +"%Y%m%d%H%M%S")

On Debian based systems:

• AIDE's configuration files are in /etc/aide/aide.conf.d/.
• You'll want to go through AIDE's documentation and the configuration files in to set them per your requirements.
• If you want new settings, to monitor a new folder for example, you'll want to add them to /etc/aide/aide.conf or /etc/aide/aide.conf.d/.
• Take a backup of the stock configuration files: sudo cp -pr /etc/aide /etc/aide-COPY-$(date +"%Y%m%d%H%M%S").

Create a new database, and install it.

On Debian based systems:

sudo aideinit

Running aide --init...
Start timestamp: 2019-04-01 21:23:37 -0400 (AIDE 0.16)
AIDE initialized database at /var/lib/aide/aide.db.new
Verbose level: 6

Number of entries:      25973

---------------------------------------------------
The attributes of the (uncompressed) database(s):
---------------------------------------------------

/var/lib/aide/aide.db.new
  RMD160   : moyQ1YskQQbidX+Lusv3g2wf1gQ=
  TIGER    : 7WoOgCrXzSpDrlO6I3PyXPj1gRiaMSeo
  SHA256   : gVx8Fp7r3800WF2aeXl+/KHCzfGsNi7O
             g16VTPpIfYQ=
  SHA512   : GYfa0DJwWgMLl4Goo5VFVOhu4BphXCo3
             rZnk49PYztwu50XjaAvsVuTjJY5uIYrG
             tV+jt3ELvwFzGefq4ZBNMg==
  CRC32    : /cusZw==
  HAVAL    : E/i5ceF3YTjwenBfyxHEsy9Kzu35VTf7
             CPGQSW4tl14=
  GOST     : n5Ityzxey9/1jIs7LMc08SULF1sLBFUc
             aMv7Oby604A=


End timestamp: 2019-04-01 21:24:45 -0400 (run time: 1m 8s)

Test everything works with no changes.

On Debian based systems:

sudo aide.wrapper --check

Start timestamp: 2019-04-01 21:24:45 -0400 (AIDE 0.16)
AIDE found NO differences between database and filesystem. Looks okay!!
Verbose level: 6

Number of entries:      25973

---------------------------------------------------
The attributes of the (uncompressed) database(s):
---------------------------------------------------

/var/lib/aide/aide.db
  RMD160   : moyQ1YskQQbidX+Lusv3g2wf1gQ=
  TIGER    : 7WoOgCrXzSpDrlO6I3PyXPj1gRiaMSeo
  SHA256   : gVx8Fp7r3800WF2aeXl+/KHCzfGsNi7O
             g16VTPpIfYQ=
  SHA512   : GYfa0DJwWgMLl4Goo5VFVOhu4BphXCo3
             rZnk49PYztwu50XjaAvsVuTjJY5uIYrG
             tV+jt3ELvwFzGefq4ZBNMg==
  CRC32    : /cusZw==
  HAVAL    : E/i5ceF3YTjwenBfyxHEsy9Kzu35VTf7
             CPGQSW4tl14=
  GOST     : n5Ityzxey9/1jIs7LMc08SULF1sLBFUc
             aMv7Oby604A=


End timestamp: 2019-04-01 21:26:03 -0400 (run time: 1m 18s)

Test everything works after making some changes.

On Debian based systems:

sudo touch /etc/test.sh
sudo touch /root/test.sh

sudo aide.wrapper --check

sudo rm /etc/test.sh
sudo rm /root/test.sh

sudo aideinit -y -f

Start timestamp: 2019-04-01 21:37:37 -0400 (AIDE 0.16)
AIDE found differences between database and filesystem!!
Verbose level: 6

Summary:
  Total number of entries:      25972
  Added entries:                2
  Removed entries:              0
  Changed entries:              1

---------------------------------------------------
Added entries:
---------------------------------------------------

f++++++++++++++++: /etc/test.sh
f++++++++++++++++: /root/test.sh

---------------------------------------------------
Changed entries:
---------------------------------------------------

d =.... mc.. .. .: /root

---------------------------------------------------
Detailed information about changes:
---------------------------------------------------

Directory: /root
  Mtime    : 2019-04-01 21:35:07 -0400        | 2019-04-01 21:37:36 -0400
  Ctime    : 2019-04-01 21:35:07 -0400        | 2019-04-01 21:37:36 -0400


---------------------------------------------------
The attributes of the (uncompressed) database(s):
---------------------------------------------------

/var/lib/aide/aide.db
  RMD160   : qF9WmKaf2PptjKnhcr9z4ueCPTY=
  TIGER    : zMo7MvvYJcq1hzvTQLPMW7ALeFiyEqv+
  SHA256   : LSLLVjjV6r8vlSxlbAbbEsPcQUB48SgP
             pdVqEn6ZNbQ=
  SHA512   : Qc4U7+ZAWCcitapGhJ1IrXCLGCf1IKZl
             02KYL1gaZ0Fm4dc7xLqjiquWDMSEbwzW
             oz49NCquqGz5jpMIUy7UxA==
  CRC32    : z8ChEA==
  HAVAL    : YapzS+/cdDwLj3kHJEq8fufLp3DPKZDg
             U12KCSkrO7Y=
  GOST     : 74sLV4HkTig+GJhokvxZQm7CJD/NR0mG
             6jV7zdt5AXQ=


End timestamp: 2019-04-01 21:38:50 -0400 (run time: 1m 13s)

That's it. If you set CRON_DAILY_RUN to yes in /etc/default/aide then cron will execute /etc/cron.daily/aide every day and e-mail you the output.

Updating The Database

Every time you make changes to files/folders that AIDE monitors, you will need to update the database to capture those changes. To do that on Debian based systems:

sudo aideinit -y -f

Anti-Virus Scanning With ClamAV (WIP)

Why

WIP

How It Works

• ClamAV is a virus scanner
• ClamAV-Freshclam is a service that keeps the virus definitions updated
• ClamAV-Daemon keeps the clamd process running to make scanning faster

Goals

WIP

Notes

• These instructions do not tell you how to enable the ClamAV daemon service to ensure clamd is running all the time. clamd is only if you're running a mail server and does not provide real-time monitoring of files. Instead, you'd want to scan files manually or on a schedule.

References

• https://www.clamav.net/documents/installation-on-debian-and-ubuntu-linux-distributions
• https://wiki.debian.org/ClamAV
• https://www.osradar.com/install-clamav-debian-9-ubuntu-18/
• https://www.lisenet.com/2014/automate-clamav-to-perform-daily-system-scan-and-send-email-notifications-on-linux/
• https://www.howtoforge.com/tutorial/configure-clamav-to-scan-and-notify-virus-and-malware/
• https://serverfault.com/questions/741299/is-there-a-way-to-keep-clamav-updated-on-debian-8
• https://askubuntu.com/questions/250290/how-do-i-scan-for-viruses-with-clamav
• https://ngothang.com/how-to-install-clamav-and-configure-daily-scanning-on-centos/

Steps

Install ClamAV.

On Debian based systems:

sudo apt install clamav clamav-freshclam clamav-daemon

Make a backup of clamav-freshclam's configuration file /etc/clamav/freshclam.conf:

sudo cp --archive /etc/clamav/freshclam.conf /etc/clamav/freshclam.conf-COPY-$(date +"%Y%m%d%H%M%S")

clamav-freshclam's default settings are probably good enough but if you want to change them, you can either edit the file /etc/clamav/freshclam.conf or use dpkg-reconfigure:

sudo dpkg-reconfigure clamav-freshclam

Note: The default settings will update the definitions 24 times in a day. To change the interval, check the Checks setting in /etc/clamav/freshclam.conf or use dpkg-reconfigure.

Start the clamav-freshclam service:

sudo service clamav-freshclam start

You can make sure clamav-freshclam running:

sudo service clamav-freshclam status

● clamav-freshclam.service - ClamAV virus database updater
   Loaded: loaded (/lib/systemd/system/clamav-freshclam.service; enabled; vendor preset: enabled)   Active: active (running) since Sat 2019-03-16 22:57:07 EDT; 2min 13s ago
     Docs: man:freshclam(1)
           man:freshclam.conf(5)
           https://www.clamav.net/documents
 Main PID: 1288 (freshclam)
   CGroup: /system.slice/clamav-freshclam.service
           └─1288 /usr/bin/freshclam -d --foreground=true

Mar 16 22:57:08 host freshclam[1288]: Sat Mar 16 22:57:08 2019 -> ^Local version: 0.100.2 Recommended version: 0.101.1
Mar 16 22:57:08 host freshclam[1288]: Sat Mar 16 22:57:08 2019 -> DON'T PANIC! Read https://www.clamav.net/documents/upgrading-clamav
Mar 16 22:57:15 host freshclam[1288]: Sat Mar 16 22:57:15 2019 -> Downloading main.cvd [100%]
Mar 16 22:57:38 host freshclam[1288]: Sat Mar 16 22:57:38 2019 -> main.cvd updated (version: 58, sigs: 4566249, f-level: 60, builder: sigmgr)
Mar 16 22:57:40 host freshclam[1288]: Sat Mar 16 22:57:40 2019 -> Downloading daily.cvd [100%]
Mar 16 22:58:13 host freshclam[1288]: Sat Mar 16 22:58:13 2019 -> daily.cvd updated (version: 25390, sigs: 1520006, f-level: 63, builder: raynman)
Mar 16 22:58:14 host freshclam[1288]: Sat Mar 16 22:58:14 2019 -> Downloading bytecode.cvd [100%]
Mar 16 22:58:16 host freshclam[1288]: Sat Mar 16 22:58:16 2019 -> bytecode.cvd updated (version: 328, sigs: 94, f-level: 63, builder: neo)
Mar 16 22:58:24 host freshclam[1288]: Sat Mar 16 22:58:24 2019 -> Database updated (6086349 signatures) from db.local.clamav.net (IP: 104.16.219.84)
Mar 16 22:58:24 host freshclam[1288]: Sat Mar 16 22:58:24 2019 -> ^Clamd was NOT notified: Can't connect to clamd through /var/run/clamav/clamd.ctl: No such file or directory

Note: Don't worry about that Local version line. Check https://serverfault.com/questions/741299/is-there-a-way-to-keep-clamav-updated-on-debian-8 for more details.

Make a backup of clamav-daemon's configuration file /etc/clamav/clamd.conf:

sudo cp --archive /etc/clamav/clamd.conf /etc/clamav/clamd.conf-COPY-$(date +"%Y%m%d%H%M%S")

You can change clamav-daemon's settings by editing the file /etc/clamav/clamd.conf or useing dpkg-reconfigure:

sudo dpkg-reconfigure clamav-daemon

Scanning Files/Folders

• To scan files/folders use the clamscan program.
• clamscan runs as the user it is executed as so it needs read permissions to the files/folders it is scanning.
• Using clamscan as root is dangerous because if a file is in fact a virus there is risk that it could use the root privileges.
• To scan a file: clamscan /path/to/file.
• To scan a directory: clamscan -r /path/to/folder.
• You can use the -i switch to only print infected files.
• Check clamscan's man pages for other switches/options.

Rootkit Detection With Rkhunter (WIP)

Why

WIP

How It Works

WIP

Goals

WIP

References

• http://rkhunter.sourceforge.net/
• https://www.cyberciti.biz/faq/howto-check-linux-rootkist-with-detectors-software/
• https://www.tecmint.com/install-rootkit-hunter-scan-for-rootkits-backdoors-in-linux/

Steps

Install Rkhunter.

On Debian based systems:

sudo apt install rkhunter

Make a backup of rkhunter' defaults file:

sudo cp -p /etc/default/rkhunter /etc/default/rkhunter-COPY-$(date +"%Y%m%d%H%M%S")

rkhunter's configuration file is /etc/rkhunter.conf. Instead of making changes to it, create and use the file /etc/rkhunter.conf.local instead:

sudo cp -p /etc/rkhunter.conf /etc/rkhunter.conf.local

Go through the configuration file /etc/rkhunter.conf.local and set to your requirements. My recommendations:

Setting	Note
UPDATE_MIRRORS=1
MIRRORS_MODE=0
MAIL-ON-WARNING=root
COPY_LOG_ON_ERROR=1	to save a copy of the log if there is an error
PKGMGR=...	set to the appropriate value per the documentation
PHALANX2_DIRTEST=1	read the documentation for why
WEB_CMD=""	this is to address an issue with the Debian package that disables the ability for rkhunter to self-update.
USE_LOCKING=1	to prevent issues with rkhunter running multiple times
SHOW_SUMMARY_WARNINGS_NUMBER=1	to see the actual number of warnings found

You want rkhunter to run every day and e-mail you the result. You can write your own script or check https://www.tecmint.com/install-rootkit-hunter-scan-for-rootkits-backdoors-in-linux/ for a sample cron script you can use.

On Debian based system, rkhunter comes with cron scripts. To enable them check /etc/default/rkhunter or use dpkg-reconfigure and say Yes to all of the questions:

sudo dpkg-reconfigure rkhunter

After you've finished with all of the changes, make sure all the settings are valid:

sudo rkhunter -C

Update rkhunter and its database:

sudo rkhunter --versioncheck
sudo rkhunter --update
sudo rkhunter --propupd

If you want to do a manual scan and see the output:

sudo rkhunter --check

Rootkit Detection With chrootkit (WIP)

Why

WIP

How It Works

WIP

Goals

WIP

References

• http://www.chkrootkit.org/
• https://www.cyberciti.biz/faq/howto-check-linux-rootkist-with-detectors-software/
• https://askubuntu.com/questions/258658/eth0-packet-sniffer-sbin-dhclient

Steps

Install chkrootkit.

On Debian based systems:

sudo apt install chkrootkit

Do a manual scan:

sudo chkrootkit

ROOTDIR is `/'
Checking `amd'...                                           not found
Checking `basename'...                                      not infected
Checking `biff'...                                          not found
Checking `chfn'...                                          not infected
Checking `chsh'...                                          not infected
...
Checking `scalper'...                                       not infected
Checking `slapper'...                                       not infected
Checking `z2'...                                            chklastlog: nothing deleted
Checking `chkutmp'...                                       chkutmp: nothing deleted
Checking `OSX_RSPLUG'...                                    not infected

Make a backup of chkrootkit's configuration file /etc/chkrootkit.conf:

sudo cp --archive /etc/chkrootkit.conf /etc/chkrootkit.conf-COPY-$(date +"%Y%m%d%H%M%S")

You want chkrootkit to run every day and e-mail you the result.

On Debian based system, chkrootkit comes with cron scripts. To enable them check /etc/chkrootkit.conf or use dpkg-reconfigure and say Yes to the first question:

sudo dpkg-reconfigure chkrootkit

logwatch - system log analyzer and reporter

Why

Your server will be generating a lot of logs that may contain important information. Unless you plan on checking your server everyday, you'll want a way to get e-mail summary of your server's logs. To accomplish this we'll use logwatch.

How It Works

logwatch scans system log files and summarizes them. You can run it directly from the command line or schedule it to run on a recurring schedule. logwatch uses service files to know how to read/summarize a log file. You can see all of the stock service files in /usr/share/logwatch/scripts/services.

logwatch's configuration file /usr/share/logwatch/default.conf/logwatch.conf specifies default options. You can override them via command line arguments.

Goals

• Logwatch configured to send a daily e-mail summary of all of the server's status and logs

Notes

• Your server will need to be able to send e-mails for this to work
• The below steps will result in logwatch running every day. If you want to change the schedule, modify the cronjob to your liking. You'll also want to change the range option to cover your recurrence window. See https://www.badpenguin.org/configure-logwatch-for-weekly-email-and-html-output-format for an example.
• If logwatch fails to deliver mail due to the e-mail having long lines please check https://blog.dhampir.no/content/exim4-line-length-in-debian-stretch-mail-delivery-failed-returning-message-to-sender as documented in issue #29. If you you followed Gmail and Exim4 As MTA With Implicit TLS then we already took care of this in step #7.

References

• Thanks to amacheema for fixing some issues with the steps and letting me know of a long line bug with exim4 as documented in issue #29.
• https://sourceforge.net/projects/logwatch/
• https://www.digitalocean.com/community/tutorials/how-to-install-and-use-logwatch-log-analyzer-and-reporter-on-a-vps

Steps

Install logwatch.

On Debian based systems:

sudo apt install logwatch

To see a sample of what logwatch collects you can run it directly:

sudo /usr/sbin/logwatch --output stdout --format text --range yesterday --service all

 ################### Logwatch 7.4.3 (12/07/16) ####################
        Processing Initiated: Mon Mar  4 00:05:50 2019
        Date Range Processed: yesterday
                              ( 2019-Mar-03 )
                              Period is day.
        Detail Level of Output: 5
        Type of Output/Format: stdout / text
        Logfiles for Host: host
 ##################################################################

 --------------------- Cron Begin ------------------------
...
...
 ---------------------- Disk Space End -------------------------


 ###################### Logwatch End #########################

Go through logwatch's self-documented configuration file /usr/share/logwatch/default.conf/logwatch.conf before continuing. There is no need to change anything here but pay special attention to the Output, Format, MailTo, Range, and Service as those are the ones we'll be using. For our purposes, instead of specifying our options in the configuration file, we will pass them as command line arguments in the daily cron job that executes logwatch. That way, if the configuration file is ever modified (e.g. during an update), our options will still be there.

Make a backup of logwatch's daily cron file /etc/cron.daily/00logwatch and unset the execute bit:

sudo cp --archive /etc/cron.daily/00logwatch /etc/cron.daily/00logwatch-COPY-$(date +"%Y%m%d%H%M%S")
sudo chmod -x /etc/cron.daily/00logwatch-COPY*

By default, logwatch outputs to stdout. Since the goal is to get a daily e-mail, we need to change the output type that logwatch uses to send e-mail instead. We could do this through the configuration file above, but that would apply to every time it is run -- even when we run it manually and want to see the output to the screen. Instead, we'll change the cron job that executes logwatch to send e-mail. This way, when run manually, we'll still get output to stdout and when run by cron, it'll send an e-mail. We'll also make sure it checks for all services, and change the output format to html so it's easier to read regardless of what the configuration file says. In the file /etc/cron.daily/00logwatch find the execute line and change it to:

/usr/sbin/logwatch --output mail --format html --mailto root --range yesterday --service all

#!/bin/bash

#Check if removed-but-not-purged
test -x /usr/share/logwatch/scripts/logwatch.pl || exit 0

#execute
/usr/sbin/logwatch --output mail --format html --mailto root --range yesterday --service all

#Note: It's possible to force the recipient in above command
#Just pass --mailto address@a.com instead of --output mail

For the lazy:

sudo sed -i -r -e "s,^($(sudo which logwatch).*?),# \1         # commented by $(whoami) on $(date +"%Y-%m-%d @ %H:%M:%S")\n$(sudo which logwatch) --output mail --format html --mailto root --range yesterday --service all         # added by $(whoami) on $(date +"%Y-%m-%d @ %H:%M:%S")," /etc/cron.daily/00logwatch

You can test the cron job by executing it:

sudo /etc/cron.daily/00logwatch

Note: If logwatch fails to deliver mail due to the e-mail having long lines please check https://blog.dhampir.no/content/exim4-line-length-in-debian-stretch-mail-delivery-failed-returning-message-to-sender as documented in issue #29. If you you followed Gmail and Exim4 As MTA With Implicit TLS then we already took care of this in step #7.

ss - Seeing Ports Your Server Is Listening On

Why

Ports are how applications, services, and processes communicate with each other -- either locally within your server or with other devices on the network. When you have an application or service (like SSH or Apache) running on your server, they listen for requests on specific ports.

Obviously we don't want your server listening on ports we don't know about. We'll use ss to see all the ports that services are listening on. This will help us track down and stop rogue, potentially dangerous, services.

Goals

• find out non-localhost what ports are open and listening for connections

References

• https://www.reddit.com/r/linux/comments/arx7st/howtosecurealinuxserver_an_evolving_howto_guide/egrib6o/
• https://www.reddit.com/r/linux/comments/arx7st/howtosecurealinuxserver_an_evolving_howto_guide/egs1rev/
• https://www.tecmint.com/find-open-ports-in-linux/
• man ss

Steps

To see the all the ports listening for traffic:

sudo ss -lntup

Netid  State      Recv-Q Send-Q     Local Address:Port     Peer Address:Port
udp    UNCONN     0      0                      *:68                  *:*        users:(("dhclient",pid=389,fd=6))
tcp    LISTEN     0      128                    *:22                  *:*        users:(("sshd",pid=4390,fd=3))
tcp    LISTEN     0      128                   :::22                 :::*        users:(("sshd",pid=4390,fd=4))

Switch Explanations:

• l = display listening sockets
• n = do now try to resolve service names
• t = display TCP sockets
• u = display UDP sockets
• p = show process information

If you see anything suspicious, like a port you're not aware of or a process you don't know, investigate and remediate as necessary.

Lynis - Linux Security Auditing

Why

From https://cisofy.com/lynis/:

Lynis is a battle-tested security tool for systems running Linux, macOS, or Unix-based operating system. It performs an extensive health scan of your systems to support system hardening and compliance testing.

Goals

• Lynis installed

Notes

• CISOFY offers packages for many distributions. Check https://packages.cisofy.com/ for distribution specific installation instructions.

References

• https://cisofy.com/documentation/lynis/get-started/
• https://packages.cisofy.com/community/#debian-ubuntu
• https://thelinuxcode.com/audit-lynis-ubuntu-server/
• https://www.vultr.com/docs/install-lynis-on-debian-8

Steps

Install lynis. https://cisofy.com/lynis/#installation has detailed instructions on how to install it for your distribution.

On Debian based systems, using CISOFY's community software repository:

sudo apt install apt-transport-https ca-certificates host
sudo wget -O - https://packages.cisofy.com/keys/cisofy-software-public.key | sudo apt-key add -
sudo echo "deb https://packages.cisofy.com/community/lynis/deb/ stable main" | sudo tee /etc/apt/sources.list.d/cisofy-lynis.list
sudo apt update
sudo apt install lynis host

Update it:

sudo lynis update info

Run a security audit:

sudo lynis audit system

This will scan your server, report its audit findings, and at the end it will give you suggestions. Spend some time going through the output and address gaps as necessary.

OSSEC - Host Intrusion Detection

Why

From https://github.com/ossec/ossec-hids

OSSEC is a full platform to monitor and control your systems. It mixes together all the aspects of HIDS (host-based intrusion detection), log monitoring and SIM/SIEM together in a simple, powerful and open source solution.

Goals

• OSSEC-HIDS installed

References

• https://www.ossec.net/docs/

Steps

Install OSSEC-HIDS from sources

sudo apt install libz-dev libssl-dev libpcre2-dev build-essential
wget https://github.com/ossec/ossec-hids/archive/3.6.0.tar.gz
tar xzf 3.6.0.tar.gz
cd ossec-hids-3.6.0/
sudo ./install.sh

Useful commands:

Agent information

 sudo /var/ossec/bin/agent_control -i <AGENT_ID>

AGENT_ID by default is 000, to be sure the command sudo /var/ossec/bin/agent_control -l can be used.

Run integrity/rootkit checking

OSSEC by default run rootkit check each 2 hours.

 sudo /var/ossec/bin/agent_control -u <AGENT_ID> -r 

Alerts

• All:

tail -f /var/ossec/logs/alerts/alerts.log

• Integrity check:

sudo cat /var/ossec/logs/alerts/alerts.log | grep -A4  -i integrity

• Rootkit check:

 sudo cat /var/ossec/logs/alerts/alerts.log | grep -A4  "rootcheck,"

The Danger Zone

Proceed At Your Own Risk

This sections cover things that are high risk because there is a possibility they can make your system unusable, or are considered unnecessary by many because the risks outweigh any rewards.

!! PROCEED AT YOUR OWN RISK !!

!! PROCEED AT YOUR OWN RISK !!

Linux Kernel sysctl Hardening

!! PROCEED AT YOUR OWN RISK !!

Why

The kernel is the brains of a Linux system. Securing it just makes sense.

Why Not

Changing kernel settings with sysctl is risky and could break your server. If you don't know what you are doing, don't have the time to debug issues, or just don't want to take the risks, I would advise from not following these steps.

Disclaimer

I am not as knowledgeable about hardening/securing a Linux kernel as I'd like. As much as I hate to admit it, I do not know what all of these settings do. My understanding is that most of them are general kernel hardening and performance, and the others are to protect against spoofing and DOS attacks.

In fact, since I am not 100% sure exactly what each setting does, I took recommended settings from numerous sites (all linked in the references below) and combined them to figure out what should be set. I figure if multiple reputable sites mention the same setting, it's probably safe.

If you have a better understanding of what these settings do, or have any other feedback/advice on them, please let me know.

I won't provide For the lazy code in this section.

Notes

• Documentation on all the sysctl settings/keys is severely lacking. The documentation I can find seems to reference the 2.2 version kernel. I could not find anything newer. If you know where I can, please let me know.
• The reference sites listed below have more comments on what each setting does.

References

• https://github.com/torvalds/linux/tree/master/Documentation
• https://www.cyberciti.biz/faq/linux-kernel-etcsysctl-conf-security-hardening/
• https://geektnt.com/sysctl-conf-hardening.html
• https://linoxide.com/how-tos/linux-server-protection/
• https://github.com/klaver/sysctl/blob/master/sysctl.conf
• https://cloudpro.zone/index.php/2018/01/30/debian-9-3-server-setup-guide-part-5/

Steps

The sysctl settings can be found in the linux-kernel-sysctl-hardening.md file in this repo.

Before you make a kernel sysctl change permanent, you can test it with the sysctl command:

sudo sysctl -w [key=value]

Example:

sudo sysctl -w kernel.ctrl-alt-del=0

Note: There are no spaces in key=value, including before and after the space.

Once you have tested a setting, and made sure it works without breaking your server, you can make it permanent by adding the values to /etc/sysctl.conf. For example:

$ sudo cat /etc/sysctl.conf
kernel.ctrl-alt-del = 0
fs.file-max = 65535
...
kernel.sysrq = 0

After updating the file you can reload the settings or reboot. To reload:

sudo sysctl -p

Note: If sysctl has trouble writing any settings then sysctl -w or sysctl -p will write an error to stderr. You can use this to quickly find invalid settings in your /etc/sysctl.conf file:

sudo sysctl -p >/dev/null

Password Protect GRUB

!! PROCEED AT YOUR OWN RISK !!

Why

If a bad actor has physical access to your server, they could use GRUB to gain unauthorized access to your system.

Why Not

If you forget the password, you'll have to go through some work to recover the password.

Goals

• auto boot the default Debian install and require a password for anything else

Notes

• This will only protect GRUB and anything behind it like your operating systems. Check your motherboard's documentation for password protecting your BIOS to prevent a bad actor from circumventing GRUB.

References

• https://selivan.github.io/2017/12/21/grub2-password-for-all-but-default-menu-entries.html
• https://help.ubuntu.com/community/Grub2/Passwords
• https://computingforgeeks.com/how-to-protect-grub-with-password-on-debian-ubuntu-and-kali-linux/
• man grub
• man grub-mkpasswd-pbkdf2

Steps

Create a Password-Based Key Derivation Function 2 (PBKDF2) hash of your password:

grub-mkpasswd-pbkdf2 -c 100000

The below output is from using password as the password:

Enter password:
Reenter password:
PBKDF2 hash of your password is grub.pbkdf2.sha512.100000.2812C233DFC899EFC3D5991D8CA74068C99D6D786A54F603E9A1EFE7BAEDDB6AA89672F92589FAF98DB9364143E7A1156C9936328971A02A483A84C3D028C4FF.C255442F9C98E1F3C500C373FE195DCF16C56EEBDC55ABDD332DD36A92865FA8FC4C90433757D743776AB186BD3AE5580F63EF445472CC1D151FA03906D08A6D

grub-mkpasswd-pbkdf2 -c 100000

Copy everything after PBKDF2 hash of your password is , starting from and including grub.pbkdf2.sha512... to the end. You'll need this in the next step.

The update-grub program uses scripts to generate configuration files it will use for GRUB's settings. Create the file /etc/grub.d/01_password and add the below code after replacing [hash] with the hash you copied from the first step. This tells update-grub to use this username and password for GRUB.

#!/bin/sh
set -e

cat << EOF
set superusers="grub"
password_pbkdf2 grub [hash]
EOF

For example:

#!/bin/sh
set -e

cat << EOF
set superusers="grub"
password_pbkdf2 grub grub.pbkdf2.sha512.100000.2812C233DFC899EFC3D5991D8CA74068C99D6D786A54F603E9A1EFE7BAEDDB6AA89672F92589FAF98DB9364143E7A1156C9936328971A02A483A84C3D028C4FF.C255442F9C98E1F3C500C373FE195DCF16C56EEBDC55ABDD332DD36A92865FA8FC4C90433757D743776AB186BD3AE5580F63EF445472CC1D151FA03906D08A6D
EOF

Set the file's execute bit so update-grub includes it when it updates GRUB's configuration:

sudo chmod a+x /etc/grub.d/01_password

Make a backup of GRUB's configuration file /etc/grub.d/10_linux that we'll be modifying and unset the execute bit so update-grub doesn't try to run it:

sudo cp --archive /etc/grub.d/10_linux /etc/grub.d/10_linux-COPY-$(date +"%Y%m%d%H%M%S")
sudo chmod a-x /etc/grub.d/10_linux.*

To make the default Debian install unrestricted (without the password) while keeping everything else restricted (with the password) modify /etc/grub.d/10_linux and add --unrestricted to the CLASS variable.

For the lazy:

sudo sed -i -r -e "/^CLASS=/ a CLASS=\"\${CLASS} --unrestricted\"         # added by $(whoami) on $(date +"%Y-%m-%d @ %H:%M:%S")" /etc/grub.d/10_linux

Update GRUB with update-grub:

sudo update-grub

 

Disable Root Login

!! PROCEED AT YOUR OWN RISK !!

Why

If you have sudo configured properly, then the root account will mostly never need to log in directly -- either at the terminal or remotely.

Why Not

Be warned, this can cause issues with some configurations!

If your installation uses sulogin (like Debian) to drop to a root console during boot failures, then locking the root account will prevent sulogin from opening the root shell and you will get this error:

Cannot open access to console, the root account is locked.

See sulogin(8) man page for more details.

Press Enter to continue.

To work around this, you can use the --force option for sulogin. Some distributions already include this, or some other, workaround.

An alternative to locking the root acount is set a long/complicated root password and store it in a secured, non digital format. That way you have it when/if you need it.

Goals

• locked root account that nobody can use to log in as root

Notes

• Some distributions disable root login by default (e.g. Ubuntu) so you may not need to do this step. Check with your distribution's documentation.

References

• https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=806852
• https://github.com/systemd/systemd/issues/7115
• https://github.com/karelzak/util-linux/commit/7ff1162e67164cb4ece19dd809c26272461aa254
• https://github.com/systemd/systemd/issues/11596
• https://www.reddit.com/r/selfhosted/comments/aoxd4l/new_guide_created_by_me_how_to_secure_a_linux/eg4rkfi/
• man systemd

Steps

Lock the root account:

sudo passwd -l root

Change Default umask

!! PROCEED AT YOUR OWN RISK !!

Why

umask controls the default permissions of files/folders when they are created. Insecure file/folder permissions give other accounts potentially unauthorized access to your data. This may include the ability to make configuration changes.

• For non-root accounts, there is no need for other accounts to get any access to the account's files/folders by default.
• For the root account, there is no need for the file/folder primary group or other accounts to have any access to root's files/folders by default.

When and if other accounts need access to a file/folder, you want to explicitly grant it using a combination of file/folder permissions and primary group.

Why Not

Changing the default umask can create unexpected problems. For example, if you set umask to 0077 for root, then non-root accounts will not have access to application configuration files/folders in /etc/ which could break applications that do not run with root privileges.

How It Works

In order to explain how umask works I'd have to explain how Linux file/folder permissions work. As that is a rather complicated question, I will defer you to the references below for further reading.

Goals

• set default umask for non-root accounts to 0027
• set default umask for the root account to 0077

Notes

• umask is a Bash built-in which means a user can change their own umask setting.

References

• https://www.linuxnix.com/umask-define-linuxunix/
• https://serverfault.com/questions/818783/which-umask-is-more-secure-in-linux-022-or-027
• https://www.cyberciti.biz/tips/understanding-linux-unix-umask-value-usage.html
• man umask

Steps

Make a backup of files we'll be editing:

sudo cp --archive /etc/profile /etc/profile-COPY-$(date +"%Y%m%d%H%M%S")
sudo cp --archive /etc/bash.bashrc /etc/bash.bashrc-COPY-$(date +"%Y%m%d%H%M%S")
sudo cp --archive /etc/login.defs /etc/login.defs-COPY-$(date +"%Y%m%d%H%M%S")
sudo cp --archive /root/.bashrc /root/.bashrc-COPY-$(date +"%Y%m%d%H%M%S")

Set default umask for non-root accounts to 0027 by adding this line to /etc/profile and /etc/bash.bashrc:

umask 0027

For the lazy:

echo -e "\numask 0027         # added by $(whoami) on $(date +"%Y-%m-%d @ %H:%M:%S")" | sudo tee -a /etc/profile /etc/bash.bashrc

We also need to add this line to /etc/login.defs:

UMASK 0027

For the lazy:

echo -e "\nUMASK 0027         # added by $(whoami) on $(date +"%Y-%m-%d @ %H:%M:%S")" | sudo tee -a /etc/login.defs

Set default umask for the root account to 0077 by adding this line to /root/.bashrc:

umask 0077

For the lazy:

echo -e "\numask 0077         # added by $(whoami) on $(date +"%Y-%m-%d @ %H:%M:%S")" | sudo tee -a /root/.bashrc

 

Orphaned Software

!! PROCEED AT YOUR OWN RISK !!

Why

As you use your system, and you install and uninstall software, you'll eventually end up with orphaned, or unused software/packages/libraries. You don't need to remove them, but if you don't need them, why keep them? When security is a priority, anything not explicitly needed is a potential security threat. You want to keep your server as trimmed and lean as possible.

Notes

• Each distribution manages software/packages/libraries differently so how you find and remove orphaned packages will be different. So far I only have steps for Debian based systems.

Debian Based Systems

On Debian based systems, you can use deborphan to find orphaned packages.

Why Not

Keep in mind, deborphan finds packages that have no package dependencies. That does not mean they are not used. You could very well have a package you use every day that has no dependencies that you wouldn't want to remove. And, if deborphan gets anything wrong, then removing critical packages may break your system.

Steps

Install deborphan.

sudo apt install deborphan

Run deborphan as root to see a list of orphaned packages:

sudo deborphan

libxapian30
libpipeline1

Assuming you want to remove all of the packages deborphan finds, you can pass it's output to apt to remove them:

sudo apt --autoremove purge $(deborphan)

 

The Miscellaneous

The Simple way with MSMTP

(#msmtp-alternative)

Why

Well I will SIMPLIFY this method, to only output email using google mail account (and others). True Simple! :)

``` bash
#!/bin/bash
###### PLEASE .... EDIT IT...
USRMAIL="usernameemail"
DOMPROV="gmail.com"
PWDEMAIL="passwordStrong"  ## ATTENTION DONT USE Special Chars.. like as SPACE # and some others not all. Feel free to test ;)
MAILPROV="smtp.google.com:583"
MYMAIL="$USRMAIL@$DOMPROV"
USERLOC="root"
#######
apt install -y msmtp
    ln -s /usr/bin/msmtp /usr/sbin/sendmail
#wget http://www.cacert.org/revoke.crl -O /etc/ssl/certs/revoke.crl
#chmod 644 /etc/ssl/certs/revoke.crl
touch /root/.msmtprc
cat <<EOF> .msmtprc
defaults
account gmail
host $MAILPROV
port $MAILPORT
#proxy_host 127.0.0.1
#proxy_port 9001
from $MYEMAIL
timeout off 
protocol smtp
#auto_from [(on|off)]
#from envelope_from
#maildomain [domain]
auth on
user $USRMAIL
passwordeval "gpg -q --for-your-eyes-only --no-tty -d /root/msmtp-mail.gpg"
#passwordeval "gpg --quiet --for-your-eyes-only --no-tty --decrypt /root/msmtp-mail.gpg"
tls on
tls_starttls on
tls_trust_file /etc/ssl/certs/ca-certificates.crt
#tls_crl_file /etc/ssl/certs/revoke.crl
#tls_fingerprint [fingerprint]
#tls_key_file [file]
#tls_cert_file [file]
tls_certcheck on
tls_force_sslv3 on
tls_min_dh_prime_bits 512
#tls_priorities [priorities]
#dsn_notify (off|condition)
#dsn_return (off|amount)
#domain argument
#keepbcc off
logfile /var/log/mail.log
syslog on
account default : gmail
EOF
chmod 0400 /root/.msmtprc

   ## In testing .. auto command
# echo -e "1\n4096\n\ny\n$MYUSRMAIL\n$MYEMAIL\nmy key\nO\n$PWDMAIL\n$PWDMAIL\n" | gpg --full-generate-key 
##
gpg --full-generate-key
gpg --output revoke.asc --gen-revoke $MYEMAIL
echo -e "$PWDEMAIL\n" | gpg -e -o /root/msmtp-mail.gpg --recipient $MYEMAIL
echo "export GPG_TTY=\$(tty)" >> .baschrc    
chmod 400 msmtp-mail.gpg

echo "Hello there" | msmtp --debug $MYEMAIL
echo"######################
## MSMTP Configured ##
######################"
```

DONE!! ;)

Gmail and Exim4 As MTA With Implicit TLS

Why

Unless you're planning on setting up your own mail server, you'll need a way to send e-mails from your server. This will be important for system alerts/messages.

You can use any Gmail account. I recommend you create one specific for this server. That way if your server is compromised, the bad-actor won't have any passwords for your primary account. Granted, if you have 2FA/MFA enabled and you use an app password, there isn't much a bad-actor can do with just the app password, but why take the risk?

There are many guides on-line that cover how to configure Gmail as MTA using STARTTLS including a previous version of this guide. With STARTTLS, an initial unencrypted connection is made and then upgraded to an encrypted TLS or SSL connection. Instead, with the approach outlined below, an encrypted TLS connection is made from the start.

Also, as discussed in issue #29 and here, exim4 will fail for messages with long lines. We'll fix this in this section too.

Goals

• mail configured to send e-mails from your server using Gmail
• long line support for exim4

References

• Thanks to remyabel for figuring out how to get this to work with TLS as documented in issue #24 and pull request #26.
• https://wiki.debian.org/Exim
• https://wiki.debian.org/GmailAndExim4
• https://www.exim.org/exim-html-current/doc/html/spec_html/ch-encrypted_smtp_connections_using_tlsssl.html
• https://php.quicoto.com/setup-exim4-to-use-gmail-in-ubuntu/
• https://www.fastmail.com/help/technical/ssltlsstarttls.html
• exim4 fails for messages with long lines - issue #29 and https://blog.dhampir.no/content/exim4-line-length-in-debian-stretch-mail-delivery-failed-returning-message-to-sender

Steps

Install exim4. You will also need openssl and ca-certificates.

On Debian based systems:

sudo apt install exim4 openssl ca-certificates

Configure exim4:

For Debian based systems:

sudo dpkg-reconfigure exim4-config

You'll be prompted with some questions:

Prompt	Answer
General type of mail configuration	mail sent by smarthost; no local mail
System mail name	localhost
IP-addresses to listen on for incoming SMTP connections	127.0.0.1; ::1
Other destinations for which mail is accepted	(default)
Visible domain name for local users	localhost
IP address or host name of the outgoing smarthost	smtp.gmail.com::465
Keep number of DNS-queries minimal (Dial-on-Demand)?	No
Split configuration into small files?	No

Make a backup of /etc/exim4/passwd.client:

sudo cp --archive /etc/exim4/passwd.client /etc/exim4/passwd.client-COPY-$(date +"%Y%m%d%H%M%S")

Add a line like this to /etc/exim4/passwd.client

Notes:

• Replace yourAccount@gmail.com and yourPassword with your details. If you have 2FA/MFA enabled on your Gmail then you'll need to create and use an app password here.
• Always check host smtp.gmail.com for the most up-to-date domains to list.

smtp.gmail.com:yourAccount@gmail.com:yourPassword
*.google.com:yourAccount@gmail.com:yourPassword

This file has your Gmail password so we need to lock it down:

sudo chown root:Debian-exim /etc/exim4/passwd.client
sudo chmod 640 /etc/exim4/passwd.client

The next step is to create an TLS certificate that exim4 will use to make the encrypted connection to smtp.gmail.com. You can use your own certificate, like one from Let's Encrypt, or create one yourself using openssl. We will use a script that comes with exim4 that calls openssl to make our certificate:

sudo bash /usr/share/doc/exim4-base/examples/exim-gencert

[*] Creating a self signed SSL certificate for Exim!
    This may be sufficient to establish encrypted connections but for
    secure identification you need to buy a real certificate!

    Please enter the hostname of your MTA at the Common Name (CN) prompt!

Generating a RSA private key
..........................................+++++
................................................+++++
writing new private key to '/etc/exim4/exim.key'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Code (2 letters) [US]:[redacted]
State or Province Name (full name) []:[redacted]
Locality Name (eg, city) []:[redacted]
Organization Name (eg, company; recommended) []:[redacted]
Organizational Unit Name (eg, section) []:[redacted]
Server name (eg. ssl.domain.tld; required!!!) []:localhost
Email Address []:[redacted]
[*] Done generating self signed certificates for exim!
    Refer to the documentation and example configuration files
    over at /usr/share/doc/exim4-base/ for an idea on how to enable TLS
    support in your mail transfer agent.

Instruct exim4 to use TLS and port 465, and fix exim4's long lines issue, by creating the file /etc/exim4/exim4.conf.localmacros and adding:

MAIN_TLS_ENABLE = 1
REMOTE_SMTP_SMARTHOST_HOSTS_REQUIRE_TLS = *
TLS_ON_CONNECT_PORTS = 465
REQUIRE_PROTOCOL = smtps
IGNORE_SMTP_LINE_LENGTH_LIMIT = true

For the lazy:

cat << EOF | sudo tee /etc/exim4/exim4.conf.localmacros
MAIN_TLS_ENABLE = 1
REMOTE_SMTP_SMARTHOST_HOSTS_REQUIRE_TLS = *
TLS_ON_CONNECT_PORTS = 465
REQUIRE_PROTOCOL = smtps
IGNORE_SMTP_LINE_LENGTH_LIMIT = true
EOF

Make a backup of exim4's configuration file /etc/exim4/exim4.conf.template:

sudo cp --archive /etc/exim4/exim4.conf.template /etc/exim4/exim4.conf.template-COPY-$(date +"%Y%m%d%H%M%S")

Add the below to /etc/exim4/exim4.conf.template after the .ifdef REMOTE_SMTP_SMARTHOST_HOSTS_REQUIRE_TLS ... .endif block:

.ifdef REQUIRE_PROTOCOL
  protocol = REQUIRE_PROTOCOL
.endif

.ifdef REMOTE_SMTP_SMARTHOST_HOSTS_REQUIRE_TLS
  hosts_require_tls = REMOTE_SMTP_SMARTHOST_HOSTS_REQUIRE_TLS
.endif
.ifdef REQUIRE_PROTOCOL
    protocol = REQUIRE_PROTOCOL
.endif
.ifdef REMOTE_SMTP_HEADERS_REWRITE
  headers_rewrite = REMOTE_SMTP_HEADERS_REWRITE
.endif

For the lazy:

sudo sed -i -r -e '/^.ifdef REMOTE_SMTP_SMARTHOST_HOSTS_REQUIRE_TLS$/I { :a; n; /^.endif$/!ba; a\# added by '"$(whoami) on $(date +"%Y-%m-%d @ %H:%M:%S")"'\n.ifdef REQUIRE_PROTOCOL\n    protocol = REQUIRE_PROTOCOL\n.endif\n# end add' -e '}' /etc/exim4/exim4.conf.template

.ifdef REQUIRE_PROTOCOL
  protocol = REQUIRE_PROTOCOL
.endif

Add the below to /etc/exim4/exim4.conf.template inside the .ifdef MAIN_TLS_ENABLE block:

.ifdef MAIN_TLS_ENABLE
.ifdef TLS_ON_CONNECT_PORTS
    tls_on_connect_ports = TLS_ON_CONNECT_PORTS
.endif

For the lazy:

sudo sed -i -r -e "/\.ifdef MAIN_TLS_ENABLE/ a # added by $(whoami) on $(date +"%Y-%m-%d @ %H:%M:%S")\n.ifdef TLS_ON_CONNECT_PORTS\n    tls_on_connect_ports = TLS_ON_CONNECT_PORTS\n.endif\n# end add" /etc/exim4/exim4.conf.template

.ifdef TLS_ON_CONNECT_PORTS
  tls_on_connect_ports = TLS_ON_CONNECT_PORTS
.endif

Update exim4 configuration to use TLS and then restart the service:

sudo update-exim4.conf
sudo service exim4 restart

If you're using UFW, you'll need to allow outbound traffic on 465. To do this we'll create a custom UFW application profile and then enable it. Create the file /etc/ufw/applications.d/smtptls, add this, then run ufw allow out smtptls comment 'open TLS port 465 for use with SMPT to send e-mails':

For the lazy:

cat << EOF | sudo tee /etc/ufw/applications.d/smtptls
[SMTPTLS]
title=SMTP through TLS
description=This opens up the TLS port 465 for use with SMPT to send e-mails.
ports=465/tcp
EOF

sudo ufw allow out smtptls comment 'open TLS port 465 for use with SMPT to send e-mails'

[SMTPTLS]
title=SMTP through TLS
description=This opens up the TLS port 465 for use with SMPT to send e-mails.
ports=465/tcp

Add some mail aliases so we can send e-mails to local accounts by adding lines like this to /etc/aliases:

You'll need to add all the local accounts that exist on your server.

user1: user1@gmail.com
user2: user2@gmail.com
...

Test your setup:

echo "test" | mail -s "Test" email@gmail.com
sudo tail /var/log/exim4/mainlog

Separate iptables Log File

Why

There will come a time when you'll need to look through your iptables logs. Having all the iptables logs go to their own file will make it a lot easier to find what you're looking for.

References

• https://blog.shadypixel.com/log-iptables-messages-to-a-separate-file-with-rsyslog/
• https://gist.github.com/netson/c45b2dc4e835761fbccc
• https://www.rsyslog.com/doc/v8-stable/configuration/actions.html

Steps

The first step is by telling your firewall to prefix all log entries with some unique string. If you're using iptables directly, you would do something like --log-prefix "[IPTABLES] " for all the rules. We took care of this in step step 4 of installing psad.

After you've added a prefix to the firewall logs, we need to tell rsyslog to send those lines to its own file. Do this by creating the file /etc/rsyslog.d/10-iptables.conf and adding this:

:msg, contains, "[IPTABLES] " /var/log/iptables.log
& stop

If you're expecting a lot if data being logged by your firewall, prefix the filename with a - "to omit syncing the file after every logging". For example:

:msg, contains, "[IPTABLES] " -/var/log/iptables.log
& stop

Note: Remember to change the prefix to whatever you use.

For the lazy:

cat << EOF | sudo tee /etc/rsyslog.d/10-iptables.conf
:msg, contains, "[IPTABLES] " /var/log/iptables.log
& stop
EOF

:msg, contains, "[IPTABLES] " -/var/log/iptables.log
& stop

:msg, contains, "[IPTABLES] " /var/log/iptables.log
& stop

Since we're logging firewall messages to a different file, we need to tell psad where the new file is. Edit /etc/psad/psad.conf and set IPT_SYSLOG_FILE to the path of the log file. For example:

Note: Remember to change the prefix to whatever you use.

For the lazy:

sudo sed -i -r -e "s/^(IPT_SYSLOG_FILE\s+)([^;]+)(;)$/# \1\2\3       # commented by $(whoami) on $(date +"%Y-%m-%d @ %H:%M:%S")\n\1\/var\/log\/iptables.log\3       # added by $(whoami) on $(date +"%Y-%m-%d @ %H:%M:%S")/" /etc/psad/psad.conf 

IPT_SYSLOG_FILE /var/log/iptables.log;

Restart psad and rsyslog to activate the changes (or reboot):

sudo psad -R
sudo psad --sig-update
sudo psad -H
sudo service rsyslog restart

The last thing we have to do is tell logrotate to rotate the new log file so it doesn't get to big and fill up our disk. Create the file /etc/logrotate.d/iptables and add this:

For the lazy:

cat << EOF | sudo tee /etc/logrotate.d/iptables
/var/log/iptables.log
{
    rotate 7
    daily
    missingok
    notifempty
    delaycompress
    compress
    postrotate
        invoke-rc.d rsyslog rotate > /dev/null
    endscript
}
EOF

/var/log/iptables.log
{
    rotate 7
    daily
    missingok
    notifempty
    delaycompress
    compress
    postrotate
        invoke-rc.d rsyslog rotate > /dev/null
    endscript
}

(Table of Contents)

Left Over

Contacting Me

For any questions, comments, concerns, feedback, or issues, submit a new issue.

(Table of Contents)

Helpful Links

• https://github.com/pratiktri/server_init_harden - Bash script that automates few of the tasks that you need to perform on a new Linux server to give it basic amount security.

(Table of Contents)

Acknowledgments

• https://www.reddit.com/r/linuxquestions/comments/aopzl7/new_guide_created_by_me_how_to_secure_a_linux/
• https://www.reddit.com/r/selfhosted/comments/aoxd4l/new_guide_created_by_me_how_to_secure_a_linux/
• https://news.ycombinator.com/item?id=19177435#19178618
• https://www.reddit.com/r/linuxadmin/comments/arx7xo/howtosecurealinuxserver_an_evolving_howto_guide/
• https://www.reddit.com/r/linux/comments/arx7st/howtosecurealinuxserver_an_evolving_howto_guide/

 

Download Details: 
Author: imthenachoman
Source Code: https://github.com/imthenachoman/How-To-Secure-A-Linux-Server 
License: CC-BY-SA-4.0 License
#linux #security 

Hire PHP Developer

Looking to develop a PHP based website from scratch or revamp your existing website?

HourlyDeveloper.io has always been an industry leader for companies and business owners looking to hire PHP web developer. By choosing to Hire PHP Developer from our company, you can always expect the best results. Our PHP services and solutions are always flexible which means that no matter the nature of your project, you can always count on us for getting the best PHP expertise.

Consult with our experts: https://bit.ly/3aEGxPy

#hire php developer #php developer #php development company #php development services #php development #php