1596962520

Queries to find the Lower Bound of K from Prefix Sum Array

Given an array A[ ] consisting of non-negative integers and matrix Q[ ][ ] consisting of queries of the following two types:

• **(1, l, val): **Update A[l] to A[l] + val.
• **(2, K): **Find the lower_bound of **K **in the prefix sum array of A[ ]. If the lower_bound does not exist print -1.

The task for each query of second type is to print the index of lower_bound of value K.

Examples:

_Input: __A[ ] = {1, 2, 3, 5, 8}, Q[ ][ ] = {{1, 0, 2}, {2, 5}, {1, 3, 5}} _

_Output: __1 _

Explanation:

Query 1: Update A[0] to A[0] + 2. Now A[ ] = {3, 2, 3, 5, 8}

_Query 2: lower_bound of K = 5 in the prefix sum array {3, 5, 8, 13, 21} is 5 and index = 1. _

Query 3: Update A[3] to A[3] + 5. Now A[ ] = {3, 2, 3, 10, 8}

_Input: __A[ ] = {4, 1, 12, 8, 20}, Q[ ] = {{2, 50}, {1, 3, 12}, {2, 50}} _

_Output: __-1 _

Recommended: Please try your approach on {IDE} first, before moving on to the solution.

Naive approach:

The simplest approach is to firstly build a prefix sum array of given array A[ ], and for queries of Type 1, update values and recalculate the prefix sum. For query of Type 2, perform a Binary Search on the prefix sum array to find lower bound.

Time Complexity:_ O(Q*(N*logn))_

_Auxiliary Space: _O(N)

Efficient Approach:

The above approach can be optimized using Fenwick Tree. Using this Data Structure, the update queries in prefix sum array can be performed in logarithmic time.

Follow the steps below to solve the problem:

• Construct the Prefix Sum Array using Fenwick Tree.
• For queries of Type 1, while** l > 0**, add val to A[l] traverse to the parent node by adding least significant bit in l.
• For queries of Type 2, perform the Binary Search on the Fenwick Tree to obtain the lower bound.
• Whenever a prefix sum greater than **K appears, **store that **index **and traverse the left part of the Fenwick Tree. Otherwise, traverse the right part of the Fenwick Tree Now, perform Binary Search.
• Finally, print the required index.

Below is the implementation of the above approach:

• Java
• C#

`// Java program to implement`

`// the above appraoch`

`**import**` `java.util.*;`

`**import**` `java.io.*;`

`**class**` `GFG {`

`// Function to calculate and return`

`// the sum of arr[0..index]`

`**static**` `**int**` `getSum(``**int**` `BITree[],`

`**int**` `index)`

`{`

`**int**` `ans =` `0``;`

`index +=` `1``;`

`// Traverse ancestors`

`// of BITree[index]`

`**while**` `(index >` `0``) {`

`// Update the sum of current`

`// element of BIT to ans`

`ans += BITree[index];`

`// Update index to that`

`// of the parent node in`

`// getSum() view by`

`// subtracting LSB(Least`

`// Significant Bit)`

`index -= index & (-index);`

`}`

`**return**` `ans;`

`}`

`// Function to update the Binary Index`

`// Tree by replacing all ancestores of`

`// index by their respective sum with val`

`**static**` `**void**` `updateBIT(``**int**` `BITree[],`

`**int**` `n,` `**int**` `index,` `**int**` `val)`

`{`

`index = index +` `1``;`

`// Traverse all ancestors`

`// and sum with 'val'.`

`**while**` `(index <= n) {`

`// Add 'val' to current`

`// node of BIT`

`BITree[index] += val;`

`// Update index to that`

`// of the parent node in`

`// updateBit() view by`

`// adding LSB(Least`

`// Significant Bit)`

`index += index & (-index);`

`}`

`}`

`// Function to construct the Binary`

`// Indexed Tree for the given array`

`**static**` `**int**``[] constructBITree(`

`**int**` `arr[],` `**int**` `n)`

`{`

`// Initialize the`

`// Binary Indexed Tree`

`**int**``[] BITree =` `**new**` `**int**``[n +` `1``];`

`**for**` `(``**int**` `i =` `0``; i <= n; i++)`

`BITree[i] =` `0``;`

`// Store the actual values in`

`// BITree[] using update()`

`**for**` `(``**int**` `i =` `0``; i < n; i++)`

`updateBIT(BITree, n, i, arr[i]);`

`**return**` `BITree;`

`}`

`// Function to obtian and return`

`// the index of lower_bound of k`

`**static**` `**int**` `getLowerBound(``**int**` `BITree[],`

`**int**``[] arr,` `**int**` `n,` `**int**` `k)`

`{`

`**int**` `lb = -``1``;`

`**int**` `l =` `0``, r = n -` `1``;`

`**while**` `(l <= r) {`

`**int**` `mid = l + (r - l) /` `2``;`

`**if**` `(getSum(BITree, mid) >= k) {`

`r = mid -` `1``;`

`lb = mid;`

`}`

`**else**`

`l = mid +` `1``;`

`}`

`**return**` `lb;`

`}`

`**static**` `**void**` `performQueries(``**int**` `A[],` `**int**` `n,` `**int**` `q[][])`

`{`

`// Store the Binary Indexed Tree`

`**int**``[] BITree = constructBITree(A, n);`

`// Solve each query in Q`

`**for**` `(``**int**` `i =` `0``; i < q.length; i++) {`

`**int**` `id = q[i][``0``];`

`**if**` `(id ==` `1``) {`

`**int**` `idx = q[i][``1``];`

`**int**` `val = q[i][``2``];`

`A[idx] += val;`

`// Update the values of all`

`// ancestors of idx`

`updateBIT(BITree, n, idx, val);`

`}`

`**else**` `{`

`**int**` `k = q[i][``1``];`

`**int**` `lb = getLowerBound(`

`BITree, A, n, k);`

`System.out.println(lb);`

`}`

`}`

`}`

`// Driver Code`

`**public**` `**static**` `**void**` `main(String[] args)`

`{`

`**int**` `A[] = {` `1``,` `2``,` `3``,` `5``,` `8` `};`

`**int**` `n = A.length;`

`**int**``[][] q = { {` `1``,` `0``,` `2` `},`

`{` `2``,` `5` `},`

`{` `1``,` `3``,` `5` `} };`

`performQueries(A, n, q);`

`}`

`}`

Output:

``````1
``````

Time Complexity:_ O(Q*(logN)2)_

Auxiliary Space:_ O(N)_

Attention reader! Don’t stop learning now. Get hold of all the important DSA concepts with the DSA Self Paced Course at a student-friendly price and become industry ready.

#advanced data structure #arrays #bit magic #mathematical #searching #array-range-queries #bit #prefix-sum

1666082925

How to Create Arrays in Python

In this tutorial, you'll know the basics of how to create arrays in Python using the array module. Learn how to use Python arrays. You'll see how to define them and the different methods commonly used for performing operations on them.

This tutorialvideo on 'Arrays in Python' will help you establish a strong hold on all the fundamentals in python programming language. Below are the topics covered in this video:
1:15 What is an array?
2:53 Is python list same as an array?
3:48  How to create arrays in python?
7:19 Accessing array elements
9:59 Basic array operations
- 10:33  Finding the length of an array
- 11:44  Adding Elements
- 15:06  Removing elements
- 18:32  Array concatenation
- 20:59  Slicing
- 23:26  Looping

Python Array Tutorial – Define, Index, Methods

In this article, you'll learn how to use Python arrays. You'll see how to define them and the different methods commonly used for performing operations on them.

The artcile covers arrays that you create by importing the `array module`. We won't cover NumPy arrays here.

1. Introduction to Arrays
1. The differences between Lists and Arrays
2. When to use arrays
2. How to use arrays
1. Define arrays
2. Find the length of arrays
3. Array indexing
4. Search through arrays
5. Loop through arrays
6. Slice an array
3. Array methods for performing operations
1. Change an existing value
2. Add a new value
3. Remove a value
4. Conclusion

Let's get started!

What are Python Arrays?

Arrays are a fundamental data structure, and an important part of most programming languages. In Python, they are containers which are able to store more than one item at the same time.

Specifically, they are an ordered collection of elements with every value being of the same data type. That is the most important thing to remember about Python arrays - the fact that they can only hold a sequence of multiple items that are of the same type.

What's the Difference between Python Lists and Python Arrays?

Lists are one of the most common data structures in Python, and a core part of the language.

Lists and arrays behave similarly.

Just like arrays, lists are an ordered sequence of elements.

They are also mutable and not fixed in size, which means they can grow and shrink throughout the life of the program. Items can be added and removed, making them very flexible to work with.

However, lists and arrays are not the same thing.

Lists store items that are of various data types. This means that a list can contain integers, floating point numbers, strings, or any other Python data type, at the same time. That is not the case with arrays.

As mentioned in the section above, arrays store only items that are of the same single data type. There are arrays that contain only integers, or only floating point numbers, or only any other Python data type you want to use.

When to Use Python Arrays

Lists are built into the Python programming language, whereas arrays aren't. Arrays are not a built-in data structure, and therefore need to be imported via the `array module` in order to be used.

Arrays of the `array module` are a thin wrapper over C arrays, and are useful when you want to work with homogeneous data.

They are also more compact and take up less memory and space which makes them more size efficient compared to lists.

If you want to perform mathematical calculations, then you should use NumPy arrays by importing the NumPy package. Besides that, you should just use Python arrays when you really need to, as lists work in a similar way and are more flexible to work with.

How to Use Arrays in Python

In order to create Python arrays, you'll first have to import the `array module` which contains all the necassary functions.

There are three ways you can import the `array module`:

• By using `import array` at the top of the file. This includes the module `array`. You would then go on to create an array using `array.array()`.
``````import array

#how you would create an array
array.array()``````
• Instead of having to type `array.array()` all the time, you could use `import array as arr` at the top of the file, instead of `import array` alone. You would then create an array by typing `arr.array()`. The `arr` acts as an alias name, with the array constructor then immediately following it.
``````import array as arr

#how you would create an array
arr.array()``````
• Lastly, you could also use `from array import *`, with `*` importing all the functionalities available. You would then create an array by writing the `array()` constructor alone.
``````from array import *

#how you would create an array
array()``````

How to Define Arrays in Python

Once you've imported the `array module`, you can then go on to define a Python array.

The general syntax for creating an array looks like this:

``variable_name = array(typecode,[elements])``

Let's break it down:

• `variable_name` would be the name of the array.
• The `typecode` specifies what kind of elements would be stored in the array. Whether it would be an array of integers, an array of floats or an array of any other Python data type. Remember that all elements should be of the same data type.
• Inside square brackets you mention the `elements` that would be stored in the array, with each element being separated by a comma. You can also create an empty array by just writing `variable_name = array(typecode)` alone, without any elements.

Below is a typecode table, with the different typecodes that can be used with the different data types when defining Python arrays:

Tying everything together, here is an example of how you would define an array in Python:

``````import array as arr

numbers = arr.array('i',[10,20,30])

print(numbers)

#output

#array('i', [10, 20, 30])``````

Let's break it down:

• First we included the array module, in this case with `import array as arr `.
• Then, we created a `numbers` array.
• We used `arr.array()` because of `import array as arr `.
• Inside the `array()` constructor, we first included `i`, for signed integer. Signed integer means that the array can include positive and negative values. Unsigned integer, with `H` for example, would mean that no negative values are allowed.
• Lastly, we included the values to be stored in the array in square brackets.

Keep in mind that if you tried to include values that were not of `i` typecode, meaning they were not integer values, you would get an error:

``````import array as arr

numbers = arr.array('i',[10.0,20,30])

print(numbers)

#output

#Traceback (most recent call last):
# File "/Users/dionysialemonaki/python_articles/demo.py", line 14, in <module>
#   numbers = arr.array('i',[10.0,20,30])
#TypeError: 'float' object cannot be interpreted as an integer``````

In the example above, I tried to include a floating point number in the array. I got an error because this is meant to be an integer array only.

Another way to create an array is the following:

``````from array import *

#an array of floating point values
numbers = array('d',[10.0,20.0,30.0])

print(numbers)

#output

#array('d', [10.0, 20.0, 30.0])``````

The example above imported the `array module` via `from array import *` and created an array `numbers` of float data type. This means that it holds only floating point numbers, which is specified with the `'d'` typecode.

How to Find the Length of an Array in Python

To find out the exact number of elements contained in an array, use the built-in `len()` method.

It will return the integer number that is equal to the total number of elements in the array you specify.

``````import array as arr

numbers = arr.array('i',[10,20,30])

print(len(numbers))

#output
# 3``````

In the example above, the array contained three elements – `10, 20, 30` – so the length of `numbers` is `3`.

Array Indexing and How to Access Individual Items in an Array in Python

Each item in an array has a specific address. Individual items are accessed by referencing their index number.

Indexing in Python, and in all programming languages and computing in general, starts at `0`. It is important to remember that counting starts at `0` and not at `1`.

To access an element, you first write the name of the array followed by square brackets. Inside the square brackets you include the item's index number.

The general syntax would look something like this:

``array_name[index_value_of_item]``

Here is how you would access each individual element in an array:

``````import array as arr

numbers = arr.array('i',[10,20,30])

print(numbers[0]) # gets the 1st element
print(numbers[1]) # gets the 2nd element
print(numbers[2]) # gets the 3rd element

#output

#10
#20
#30``````

Remember that the index value of the last element of an array is always one less than the length of the array. Where `n` is the length of the array, `n - 1` will be the index value of the last item.

Note that you can also access each individual element using negative indexing.

With negative indexing, the last element would have an index of `-1`, the second to last element would have an index of `-2`, and so on.

Here is how you would get each item in an array using that method:

``````import array as arr

numbers = arr.array('i',[10,20,30])

print(numbers[-1]) #gets last item
print(numbers[-2]) #gets second to last item
print(numbers[-3]) #gets first item

#output

#30
#20
#10``````

How to Search Through an Array in Python

You can find out an element's index number by using the `index()` method.

You pass the value of the element being searched as the argument to the method, and the element's index number is returned.

``````import array as arr

numbers = arr.array('i',[10,20,30])

#search for the index of the value 10
print(numbers.index(10))

#output

#0``````

If there is more than one element with the same value, the index of the first instance of the value will be returned:

``````import array as arr

numbers = arr.array('i',[10,20,30,10,20,30])

#search for the index of the value 10
#will return the index number of the first instance of the value 10
print(numbers.index(10))

#output

#0``````

How to Loop through an Array in Python

You've seen how to access each individual element in an array and print it out on its own.

You've also seen how to print the array, using the `print()` method. That method gives the following result:

``````import array as arr

numbers = arr.array('i',[10,20,30])

print(numbers)

#output

#array('i', [10, 20, 30])``````

What if you want to print each value one by one?

This is where a loop comes in handy. You can loop through the array and print out each value, one-by-one, with each loop iteration.

For this you can use a simple `for` loop:

``````import array as arr

numbers = arr.array('i',[10,20,30])

for number in numbers:
print(number)

#output
#10
#20
#30``````

You could also use the `range()` function, and pass the `len()` method as its parameter. This would give the same result as above:

``````import array as arr

values = arr.array('i',[10,20,30])

#prints each individual value in the array
for value in range(len(values)):
print(values[value])

#output

#10
#20
#30``````

How to Slice an Array in Python

To access a specific range of values inside the array, use the slicing operator, which is a colon `:`.

When using the slicing operator and you only include one value, the counting starts from `0` by default. It gets the first item, and goes up to but not including the index number you specify.

``````import array as arr

#original array
numbers = arr.array('i',[10,20,30])

#get the values 10 and 20 only
print(numbers[:2])  #first to second position

#output

#array('i', [10, 20])``````

When you pass two numbers as arguments, you specify a range of numbers. In this case, the counting starts at the position of the first number in the range, and up to but not including the second one:

``````import array as arr

#original array
numbers = arr.array('i',[10,20,30])

#get the values 20 and 30 only
print(numbers[1:3]) #second to third position

#output

#rray('i', [20, 30])``````

Methods For Performing Operations on Arrays in Python

Arrays are mutable, which means they are changeable. You can change the value of the different items, add new ones, or remove any you don't want in your program anymore.

Let's see some of the most commonly used methods which are used for performing operations on arrays.

How to Change the Value of an Item in an Array

You can change the value of a specific element by speficying its position and assigning it a new value:

``````import array as arr

#original array
numbers = arr.array('i',[10,20,30])

#change the first element
#change it from having a value of 10 to having a value of 40
numbers[0] = 40

print(numbers)

#output

#array('i', [40, 20, 30])``````

How to Add a New Value to an Array

To add one single value at the end of an array, use the `append()` method:

``````import array as arr

#original array
numbers = arr.array('i',[10,20,30])

#add the integer 40 to the end of numbers
numbers.append(40)

print(numbers)

#output

#array('i', [10, 20, 30, 40])``````

Be aware that the new item you add needs to be the same data type as the rest of the items in the array.

Look what happens when I try to add a float to an array of integers:

``````import array as arr

#original array
numbers = arr.array('i',[10,20,30])

#add the integer 40 to the end of numbers
numbers.append(40.0)

print(numbers)

#output

#Traceback (most recent call last):
#  File "/Users/dionysialemonaki/python_articles/demo.py", line 19, in <module>
#   numbers.append(40.0)
#TypeError: 'float' object cannot be interpreted as an integer``````

But what if you want to add more than one value to the end an array?

Use the `extend()` method, which takes an iterable (such as a list of items) as an argument. Again, make sure that the new items are all the same data type.

``````import array as arr

#original array
numbers = arr.array('i',[10,20,30])

#add the integers 40,50,60 to the end of numbers
#The numbers need to be enclosed in square brackets

numbers.extend([40,50,60])

print(numbers)

#output

#array('i', [10, 20, 30, 40, 50, 60])``````

And what if you don't want to add an item to the end of an array? Use the `insert()` method, to add an item at a specific position.

The `insert()` function takes two arguments: the index number of the position the new element will be inserted, and the value of the new element.

``````import array as arr

#original array
numbers = arr.array('i',[10,20,30])

#add the integer 40 in the first position
#remember indexing starts at 0

numbers.insert(0,40)

print(numbers)

#output

#array('i', [40, 10, 20, 30])``````

How to Remove a Value from an Array

To remove an element from an array, use the `remove()` method and include the value as an argument to the method.

``````import array as arr

#original array
numbers = arr.array('i',[10,20,30])

numbers.remove(10)

print(numbers)

#output

#array('i', [20, 30])``````

With `remove()`, only the first instance of the value you pass as an argument will be removed.

See what happens when there are more than one identical values:

``````import array as arr

#original array
numbers = arr.array('i',[10,20,30,10,20])

numbers.remove(10)

print(numbers)

#output

#array('i', [20, 30, 10, 20])``````

Only the first occurence of `10` is removed.

You can also use the `pop()` method, and specify the position of the element to be removed:

``````import array as arr

#original array
numbers = arr.array('i',[10,20,30,10,20])

#remove the first instance of 10
numbers.pop(0)

print(numbers)

#output

#array('i', [20, 30, 10, 20])``````

Conclusion

And there you have it - you now know the basics of how to create arrays in Python using the `array module`. Hopefully you found this guide helpful.

Thanks for reading and happy coding!

#python #programming

1670560264

Understanding Arrays in Python

Learn how to use Python arrays. Create arrays in Python using the array module. You'll see how to define them and the different methods commonly used for performing operations on them.

The artcile covers arrays that you create by importing the `array module`. We won't cover NumPy arrays here.

1. Introduction to Arrays
1. The differences between Lists and Arrays
2. When to use arrays
2. How to use arrays
1. Define arrays
2. Find the length of arrays
3. Array indexing
4. Search through arrays
5. Loop through arrays
6. Slice an array
3. Array methods for performing operations
1. Change an existing value
2. Add a new value
3. Remove a value
4. Conclusion

Let's get started!

What are Python Arrays?

Arrays are a fundamental data structure, and an important part of most programming languages. In Python, they are containers which are able to store more than one item at the same time.

Specifically, they are an ordered collection of elements with every value being of the same data type. That is the most important thing to remember about Python arrays - the fact that they can only hold a sequence of multiple items that are of the same type.

What's the Difference between Python Lists and Python Arrays?

Lists are one of the most common data structures in Python, and a core part of the language.

Lists and arrays behave similarly.

Just like arrays, lists are an ordered sequence of elements.

They are also mutable and not fixed in size, which means they can grow and shrink throughout the life of the program. Items can be added and removed, making them very flexible to work with.

However, lists and arrays are not the same thing.

Lists store items that are of various data types. This means that a list can contain integers, floating point numbers, strings, or any other Python data type, at the same time. That is not the case with arrays.

As mentioned in the section above, arrays store only items that are of the same single data type. There are arrays that contain only integers, or only floating point numbers, or only any other Python data type you want to use.

When to Use Python Arrays

Lists are built into the Python programming language, whereas arrays aren't. Arrays are not a built-in data structure, and therefore need to be imported via the `array module` in order to be used.

Arrays of the `array module` are a thin wrapper over C arrays, and are useful when you want to work with homogeneous data.

They are also more compact and take up less memory and space which makes them more size efficient compared to lists.

If you want to perform mathematical calculations, then you should use NumPy arrays by importing the NumPy package. Besides that, you should just use Python arrays when you really need to, as lists work in a similar way and are more flexible to work with.

How to Use Arrays in Python

In order to create Python arrays, you'll first have to import the `array module` which contains all the necassary functions.

There are three ways you can import the `array module`:

1. By using `import array` at the top of the file. This includes the module `array`. You would then go on to create an array using `array.array()`.
``````import array

#how you would create an array
array.array()
``````
1. Instead of having to type `array.array()` all the time, you could use `import array as arr` at the top of the file, instead of `import array` alone. You would then create an array by typing `arr.array()`. The `arr` acts as an alias name, with the array constructor then immediately following it.
``````import array as arr

#how you would create an array
arr.array()
``````
1. Lastly, you could also use `from array import *`, with `*` importing all the functionalities available. You would then create an array by writing the `array()` constructor alone.
``````from array import *

#how you would create an array
array()
``````

How to Define Arrays in Python

Once you've imported the `array module`, you can then go on to define a Python array.

The general syntax for creating an array looks like this:

``````variable_name = array(typecode,[elements])
``````

Let's break it down:

• `variable_name` would be the name of the array.
• The `typecode` specifies what kind of elements would be stored in the array. Whether it would be an array of integers, an array of floats or an array of any other Python data type. Remember that all elements should be of the same data type.
• Inside square brackets you mention the `elements` that would be stored in the array, with each element being separated by a comma. You can also create an empty array by just writing `variable_name = array(typecode)` alone, without any elements.

Below is a typecode table, with the different typecodes that can be used with the different data types when defining Python arrays:

Tying everything together, here is an example of how you would define an array in Python:

``````import array as arr

numbers = arr.array('i',[10,20,30])

print(numbers)

#output

#array('i', [10, 20, 30])
``````

Let's break it down:

• First we included the array module, in this case with `import array as arr `.
• Then, we created a `numbers` array.
• We used `arr.array()` because of `import array as arr `.
• Inside the `array()` constructor, we first included `i`, for signed integer. Signed integer means that the array can include positive and negative values. Unsigned integer, with `H` for example, would mean that no negative values are allowed.
• Lastly, we included the values to be stored in the array in square brackets.

Keep in mind that if you tried to include values that were not of `i` typecode, meaning they were not integer values, you would get an error:

``````import array as arr

numbers = arr.array('i',[10.0,20,30])

print(numbers)

#output

#Traceback (most recent call last):
# File "/Users/dionysialemonaki/python_articles/demo.py", line 14, in <module>
#   numbers = arr.array('i',[10.0,20,30])
#TypeError: 'float' object cannot be interpreted as an integer
``````

In the example above, I tried to include a floating point number in the array. I got an error because this is meant to be an integer array only.

Another way to create an array is the following:

``````from array import *

#an array of floating point values
numbers = array('d',[10.0,20.0,30.0])

print(numbers)

#output

#array('d', [10.0, 20.0, 30.0])
``````

The example above imported the `array module` via `from array import *` and created an array `numbers` of float data type. This means that it holds only floating point numbers, which is specified with the `'d'` typecode.

How to Find the Length of an Array in Python

To find out the exact number of elements contained in an array, use the built-in `len()` method.

It will return the integer number that is equal to the total number of elements in the array you specify.

``````import array as arr

numbers = arr.array('i',[10,20,30])

print(len(numbers))

#output
# 3
``````

In the example above, the array contained three elements – `10, 20, 30` – so the length of `numbers` is `3`.

Array Indexing and How to Access Individual Items in an Array in Python

Each item in an array has a specific address. Individual items are accessed by referencing their index number.

Indexing in Python, and in all programming languages and computing in general, starts at `0`. It is important to remember that counting starts at `0` and not at `1`.

To access an element, you first write the name of the array followed by square brackets. Inside the square brackets you include the item's index number.

The general syntax would look something like this:

``````array_name[index_value_of_item]
``````

Here is how you would access each individual element in an array:

``````import array as arr

numbers = arr.array('i',[10,20,30])

print(numbers[0]) # gets the 1st element
print(numbers[1]) # gets the 2nd element
print(numbers[2]) # gets the 3rd element

#output

#10
#20
#30
``````

Remember that the index value of the last element of an array is always one less than the length of the array. Where `n` is the length of the array, `n - 1` will be the index value of the last item.

Note that you can also access each individual element using negative indexing.

With negative indexing, the last element would have an index of `-1`, the second to last element would have an index of `-2`, and so on.

Here is how you would get each item in an array using that method:

``````import array as arr

numbers = arr.array('i',[10,20,30])

print(numbers[-1]) #gets last item
print(numbers[-2]) #gets second to last item
print(numbers[-3]) #gets first item

#output

#30
#20
#10
``````

How to Search Through an Array in Python

You can find out an element's index number by using the `index()` method.

You pass the value of the element being searched as the argument to the method, and the element's index number is returned.

``````import array as arr

numbers = arr.array('i',[10,20,30])

#search for the index of the value 10
print(numbers.index(10))

#output

#0
``````

If there is more than one element with the same value, the index of the first instance of the value will be returned:

``````import array as arr

numbers = arr.array('i',[10,20,30,10,20,30])

#search for the index of the value 10
#will return the index number of the first instance of the value 10
print(numbers.index(10))

#output

#0
``````

How to Loop through an Array in Python

You've seen how to access each individual element in an array and print it out on its own.

You've also seen how to print the array, using the `print()` method. That method gives the following result:

``````import array as arr

numbers = arr.array('i',[10,20,30])

print(numbers)

#output

#array('i', [10, 20, 30])
``````

What if you want to print each value one by one?

This is where a loop comes in handy. You can loop through the array and print out each value, one-by-one, with each loop iteration.

For this you can use a simple `for` loop:

``````import array as arr

numbers = arr.array('i',[10,20,30])

for number in numbers:
print(number)

#output
#10
#20
#30
``````

You could also use the `range()` function, and pass the `len()` method as its parameter. This would give the same result as above:

``````import array as arr

values = arr.array('i',[10,20,30])

#prints each individual value in the array
for value in range(len(values)):
print(values[value])

#output

#10
#20
#30
``````

How to Slice an Array in Python

To access a specific range of values inside the array, use the slicing operator, which is a colon `:`.

When using the slicing operator and you only include one value, the counting starts from `0` by default. It gets the first item, and goes up to but not including the index number you specify.

``````
import array as arr

#original array
numbers = arr.array('i',[10,20,30])

#get the values 10 and 20 only
print(numbers[:2])  #first to second position

#output

#array('i', [10, 20])
``````

When you pass two numbers as arguments, you specify a range of numbers. In this case, the counting starts at the position of the first number in the range, and up to but not including the second one:

``````import array as arr

#original array
numbers = arr.array('i',[10,20,30])

#get the values 20 and 30 only
print(numbers[1:3]) #second to third position

#output

#rray('i', [20, 30])
``````

Methods For Performing Operations on Arrays in Python

Arrays are mutable, which means they are changeable. You can change the value of the different items, add new ones, or remove any you don't want in your program anymore.

Let's see some of the most commonly used methods which are used for performing operations on arrays.

How to Change the Value of an Item in an Array

You can change the value of a specific element by speficying its position and assigning it a new value:

``````import array as arr

#original array
numbers = arr.array('i',[10,20,30])

#change the first element
#change it from having a value of 10 to having a value of 40
numbers[0] = 40

print(numbers)

#output

#array('i', [40, 20, 30])
``````

How to Add a New Value to an Array

To add one single value at the end of an array, use the `append()` method:

``````import array as arr

#original array
numbers = arr.array('i',[10,20,30])

#add the integer 40 to the end of numbers
numbers.append(40)

print(numbers)

#output

#array('i', [10, 20, 30, 40])
``````

Be aware that the new item you add needs to be the same data type as the rest of the items in the array.

Look what happens when I try to add a float to an array of integers:

``````import array as arr

#original array
numbers = arr.array('i',[10,20,30])

#add the integer 40 to the end of numbers
numbers.append(40.0)

print(numbers)

#output

#Traceback (most recent call last):
#  File "/Users/dionysialemonaki/python_articles/demo.py", line 19, in <module>
#   numbers.append(40.0)
#TypeError: 'float' object cannot be interpreted as an integer
``````

But what if you want to add more than one value to the end an array?

Use the `extend()` method, which takes an iterable (such as a list of items) as an argument. Again, make sure that the new items are all the same data type.

``````import array as arr

#original array
numbers = arr.array('i',[10,20,30])

#add the integers 40,50,60 to the end of numbers
#The numbers need to be enclosed in square brackets

numbers.extend([40,50,60])

print(numbers)

#output

#array('i', [10, 20, 30, 40, 50, 60])
``````

And what if you don't want to add an item to the end of an array? Use the `insert()` method, to add an item at a specific position.

The `insert()` function takes two arguments: the index number of the position the new element will be inserted, and the value of the new element.

``````import array as arr

#original array
numbers = arr.array('i',[10,20,30])

#add the integer 40 in the first position
#remember indexing starts at 0

numbers.insert(0,40)

print(numbers)

#output

#array('i', [40, 10, 20, 30])
``````

How to Remove a Value from an Array

To remove an element from an array, use the `remove()` method and include the value as an argument to the method.

``````import array as arr

#original array
numbers = arr.array('i',[10,20,30])

numbers.remove(10)

print(numbers)

#output

#array('i', [20, 30])
``````

With `remove()`, only the first instance of the value you pass as an argument will be removed.

See what happens when there are more than one identical values:

``````
import array as arr

#original array
numbers = arr.array('i',[10,20,30,10,20])

numbers.remove(10)

print(numbers)

#output

#array('i', [20, 30, 10, 20])
``````

Only the first occurence of `10` is removed.

You can also use the `pop()` method, and specify the position of the element to be removed:

``````import array as arr

#original array
numbers = arr.array('i',[10,20,30,10,20])

#remove the first instance of 10
numbers.pop(0)

print(numbers)

#output

#array('i', [20, 30, 10, 20])
``````

Conclusion

And there you have it - you now know the basics of how to create arrays in Python using the `array module`. Hopefully you found this guide helpful.

You'll start from the basics and learn in an interacitve and beginner-friendly way. You'll also build five projects at the end to put into practice and help reinforce what you learned.

Thanks for reading and happy coding!

Original article source at https://www.freecodecamp.org

#python

1596962520

Queries to find the Lower Bound of K from Prefix Sum Array

Given an array A[ ] consisting of non-negative integers and matrix Q[ ][ ] consisting of queries of the following two types:

• **(1, l, val): **Update A[l] to A[l] + val.
• **(2, K): **Find the lower_bound of **K **in the prefix sum array of A[ ]. If the lower_bound does not exist print -1.

The task for each query of second type is to print the index of lower_bound of value K.

Examples:

_Input: __A[ ] = {1, 2, 3, 5, 8}, Q[ ][ ] = {{1, 0, 2}, {2, 5}, {1, 3, 5}} _

_Output: __1 _

Explanation:

Query 1: Update A[0] to A[0] + 2. Now A[ ] = {3, 2, 3, 5, 8}

_Query 2: lower_bound of K = 5 in the prefix sum array {3, 5, 8, 13, 21} is 5 and index = 1. _

Query 3: Update A[3] to A[3] + 5. Now A[ ] = {3, 2, 3, 10, 8}

_Input: __A[ ] = {4, 1, 12, 8, 20}, Q[ ] = {{2, 50}, {1, 3, 12}, {2, 50}} _

_Output: __-1 _

Recommended: Please try your approach on {IDE} first, before moving on to the solution.

Naive approach:

The simplest approach is to firstly build a prefix sum array of given array A[ ], and for queries of Type 1, update values and recalculate the prefix sum. For query of Type 2, perform a Binary Search on the prefix sum array to find lower bound.

Time Complexity:_ O(Q*(N*logn))_

_Auxiliary Space: _O(N)

Efficient Approach:

The above approach can be optimized using Fenwick Tree. Using this Data Structure, the update queries in prefix sum array can be performed in logarithmic time.

Follow the steps below to solve the problem:

• Construct the Prefix Sum Array using Fenwick Tree.
• For queries of Type 1, while** l > 0**, add val to A[l] traverse to the parent node by adding least significant bit in l.
• For queries of Type 2, perform the Binary Search on the Fenwick Tree to obtain the lower bound.
• Whenever a prefix sum greater than **K appears, **store that **index **and traverse the left part of the Fenwick Tree. Otherwise, traverse the right part of the Fenwick Tree Now, perform Binary Search.
• Finally, print the required index.

Below is the implementation of the above approach:

• Java
• C#

`// Java program to implement`

`// the above appraoch`

`**import**` `java.util.*;`

`**import**` `java.io.*;`

`**class**` `GFG {`

`// Function to calculate and return`

`// the sum of arr[0..index]`

`**static**` `**int**` `getSum(``**int**` `BITree[],`

`**int**` `index)`

`{`

`**int**` `ans =` `0``;`

`index +=` `1``;`

`// Traverse ancestors`

`// of BITree[index]`

`**while**` `(index >` `0``) {`

`// Update the sum of current`

`// element of BIT to ans`

`ans += BITree[index];`

`// Update index to that`

`// of the parent node in`

`// getSum() view by`

`// subtracting LSB(Least`

`// Significant Bit)`

`index -= index & (-index);`

`}`

`**return**` `ans;`

`}`

`// Function to update the Binary Index`

`// Tree by replacing all ancestores of`

`// index by their respective sum with val`

`**static**` `**void**` `updateBIT(``**int**` `BITree[],`

`**int**` `n,` `**int**` `index,` `**int**` `val)`

`{`

`index = index +` `1``;`

`// Traverse all ancestors`

`// and sum with 'val'.`

`**while**` `(index <= n) {`

`// Add 'val' to current`

`// node of BIT`

`BITree[index] += val;`

`// Update index to that`

`// of the parent node in`

`// updateBit() view by`

`// adding LSB(Least`

`// Significant Bit)`

`index += index & (-index);`

`}`

`}`

`// Function to construct the Binary`

`// Indexed Tree for the given array`

`**static**` `**int**``[] constructBITree(`

`**int**` `arr[],` `**int**` `n)`

`{`

`// Initialize the`

`// Binary Indexed Tree`

`**int**``[] BITree =` `**new**` `**int**``[n +` `1``];`

`**for**` `(``**int**` `i =` `0``; i <= n; i++)`

`BITree[i] =` `0``;`

`// Store the actual values in`

`// BITree[] using update()`

`**for**` `(``**int**` `i =` `0``; i < n; i++)`

`updateBIT(BITree, n, i, arr[i]);`

`**return**` `BITree;`

`}`

`// Function to obtian and return`

`// the index of lower_bound of k`

`**static**` `**int**` `getLowerBound(``**int**` `BITree[],`

`**int**``[] arr,` `**int**` `n,` `**int**` `k)`

`{`

`**int**` `lb = -``1``;`

`**int**` `l =` `0``, r = n -` `1``;`

`**while**` `(l <= r) {`

`**int**` `mid = l + (r - l) /` `2``;`

`**if**` `(getSum(BITree, mid) >= k) {`

`r = mid -` `1``;`

`lb = mid;`

`}`

`**else**`

`l = mid +` `1``;`

`}`

`**return**` `lb;`

`}`

`**static**` `**void**` `performQueries(``**int**` `A[],` `**int**` `n,` `**int**` `q[][])`

`{`

`// Store the Binary Indexed Tree`

`**int**``[] BITree = constructBITree(A, n);`

`// Solve each query in Q`

`**for**` `(``**int**` `i =` `0``; i < q.length; i++) {`

`**int**` `id = q[i][``0``];`

`**if**` `(id ==` `1``) {`

`**int**` `idx = q[i][``1``];`

`**int**` `val = q[i][``2``];`

`A[idx] += val;`

`// Update the values of all`

`// ancestors of idx`

`updateBIT(BITree, n, idx, val);`

`}`

`**else**` `{`

`**int**` `k = q[i][``1``];`

`**int**` `lb = getLowerBound(`

`BITree, A, n, k);`

`System.out.println(lb);`

`}`

`}`

`}`

`// Driver Code`

`**public**` `**static**` `**void**` `main(String[] args)`

`{`

`**int**` `A[] = {` `1``,` `2``,` `3``,` `5``,` `8` `};`

`**int**` `n = A.length;`

`**int**``[][] q = { {` `1``,` `0``,` `2` `},`

`{` `2``,` `5` `},`

`{` `1``,` `3``,` `5` `} };`

`performQueries(A, n, q);`

`}`

`}`

Output:

``````1
``````

Time Complexity:_ O(Q*(logN)2)_

Auxiliary Space:_ O(N)_

Attention reader! Don’t stop learning now. Get hold of all the important DSA concepts with the DSA Self Paced Course at a student-friendly price and become industry ready.

#advanced data structure #arrays #bit magic #mathematical #searching #array-range-queries #bit #prefix-sum

1659283860

Activeinteraction: Manage Application Specific Business Logic Of Ruby

ActiveInteraction

ActiveInteraction manages application-specific business logic. It's an implementation of service objects designed to blend seamlessly into Rails.

ActiveInteraction gives you a place to put your business logic. It also helps you write safer code by validating that your inputs conform to your expectations. If ActiveModel deals with your nouns, then ActiveInteraction handles your verbs.

API Documentation

Installation

``````gem 'active_interaction', '~> 5.1'
``````

Or install it manually:

``````\$ gem install active_interaction --version '~> 5.1'
``````

This project uses Semantic Versioning. Check out GitHub releases for a detailed list of changes.

Basic usage

To define an interaction, create a subclass of `ActiveInteraction::Base`. Then you need to do two things:

Define your inputs. Use class filter methods to define what you expect your inputs to look like. For instance, if you need a boolean flag for pepperoni, use `boolean :pepperoni`. Check out the filters section for all the available options.

Define your business logic. Do this by implementing the `#execute` method. Each input you defined will be available as the type you specified. If any of the inputs are invalid, `#execute` won't be run. Filters are responsible for checking your inputs. Check out the validations section if you need more than that.

That covers the basics. Let's put it all together into a simple example that squares a number.

``````require 'active_interaction'

class Square < ActiveInteraction::Base
float :x

def execute
x**2
end
end
``````

Call `.run` on your interaction to execute it. You must pass a single hash to `.run`. It will return an instance of your interaction. By convention, we call this an outcome. You can use the `#valid?` method to ask the outcome if it's valid. If it's invalid, take a look at its errors with `#errors`. In either case, the value returned from `#execute` will be stored in `#result`.

``````outcome = Square.run(x: 'two point one')
outcome.valid?
# => nil
outcome.errors.messages
# => {:x=>["is not a valid float"]}

outcome = Square.run(x: 2.1)
outcome.valid?
# => true
outcome.result
# => 4.41
``````

You can also use `.run!` to execute interactions. It's like `.run` but more dangerous. It doesn't return an outcome. If the outcome would be invalid, it will instead raise an error. But if the outcome would be valid, it simply returns the result.

``````Square.run!(x: 'two point one')
# ActiveInteraction::InvalidInteractionError: X is not a valid float
Square.run!(x: 2.1)
# => 4.41
``````

Validations

ActiveInteraction checks your inputs. Often you'll want more than that. For instance, you may want an input to be a string with at least one non-whitespace character. Instead of writing your own validation for that, you can use validations from ActiveModel.

These validations aren't provided by ActiveInteraction. They're from ActiveModel. You can also use any custom validations you wrote yourself in your interactions.

``````class SayHello < ActiveInteraction::Base
string :name

validates :name,
presence: true

def execute
"Hello, #{name}!"
end
end
``````

When you run this interaction, two things will happen. First ActiveInteraction will check your inputs. Then ActiveModel will validate them. If both of those are happy, it will be executed.

``````SayHello.run!(name: nil)
# ActiveInteraction::InvalidInteractionError: Name is required

SayHello.run!(name: '')
# ActiveInteraction::InvalidInteractionError: Name can't be blank

SayHello.run!(name: 'Taylor')
# => "Hello, Taylor!"
``````

Filters

You can define filters inside an interaction using the appropriate class method. Each method has the same signature:

Some symbolic names. These are the attributes to create.

An optional hash of options. Each filter supports at least these two options:

`default` is the fallback value to use if `nil` is given. To make a filter optional, set `default: nil`.

`desc` is a human-readable description of the input. This can be useful for generating documentation. For more information about this, read the descriptions section.

An optional block of sub-filters. Only array and hash filters support this. Other filters will ignore blocks when given to them.

Let's take a look at an example filter. It defines three inputs: `x`, `y`, and `z`. Those inputs are optional and they all share the same description ("an example filter").

``````array :x, :y, :z,
default: nil,
desc: 'an example filter' do
# Some filters support sub-filters here.
end
``````

In general, filters accept values of the type they correspond to, plus a few alternatives that can be reasonably coerced. Typically the coercions come from Rails, so `"1"` can be interpreted as the boolean value `true`, the string `"1"`, or the number `1`.

Basic Filters

Array

In addition to accepting arrays, array inputs will convert `ActiveRecord::Relation`s into arrays.

``````class ArrayInteraction < ActiveInteraction::Base
array :toppings

def execute
toppings.size
end
end

ArrayInteraction.run!(toppings: 'everything')
# ActiveInteraction::InvalidInteractionError: Toppings is not a valid array
ArrayInteraction.run!(toppings: [:cheese, 'pepperoni'])
# => 2
``````

Use a block to constrain the types of elements an array can contain. Note that you can only have one filter inside an array block, and it must not have a name.

``````array :birthdays do
date
end
``````

For `interface`, `object`, and `record` filters, the name of the array filter will be singularized and used to determine the type of value passed. In the example below, the objects passed would need to be of type `Cow`.

``````array :cows do
object
end
``````

You can override this by passing the necessary information to the inner filter.

``````array :managers do
object class: People
end
``````

Errors that occur will be indexed based on the Rails configuration setting `index_nested_attribute_errors`. You can also manually override this setting with the `:index_errors` option. In this state is is possible to get multiple errors from a single filter.

``````class ArrayInteraction < ActiveInteraction::Base
array :favorite_numbers, index_errors: true do
integer
end

def execute
favorite_numbers
end
end

ArrayInteraction.run(favorite_numbers: [8, 'bazillion']).errors.details
=> {:"favorite_numbers[1]"=>[{:error=>:invalid_type, :type=>"array"}]}
``````

With `:index_errors` set to `false` the error would have been:

``````{:favorite_numbers=>[{:error=>:invalid_type, :type=>"array"}]}
``````

Boolean

Boolean filters convert the strings `"1"`, `"true"`, and `"on"` (case-insensitive) into `true`. They also convert `"0"`, `"false"`, and `"off"` into `false`. Blank strings will be treated as `nil`.

``````class BooleanInteraction < ActiveInteraction::Base
boolean :kool_aid

def execute
'Oh yeah!' if kool_aid
end
end

BooleanInteraction.run!(kool_aid: 1)
# ActiveInteraction::InvalidInteractionError: Kool aid is not a valid boolean
BooleanInteraction.run!(kool_aid: true)
# => "Oh yeah!"
``````

File

File filters also accept `TempFile`s and anything that responds to `#rewind`. That means that you can pass the `params` from uploading files via forms in Rails.

``````class FileInteraction < ActiveInteraction::Base

def execute
end
end

# ActiveInteraction::InvalidInteractionError: Readme is not a valid file
# => 21563
``````

Hash

Hash filters accept hashes. The expected value types are given by passing a block and nesting other filters. You can have any number of filters inside a hash, including other hashes.

``````class HashInteraction < ActiveInteraction::Base
hash :preferences do
boolean :sweepstakes
end

def execute
puts 'Thanks for joining the newsletter!' if preferences[:newsletter]
puts 'Good luck in the sweepstakes!' if preferences[:sweepstakes]
end
end

HashInteraction.run!(preferences: 'yes, no')
# ActiveInteraction::InvalidInteractionError: Preferences is not a valid hash
HashInteraction.run!(preferences: { newsletter: true, 'sweepstakes' => false })
# Thanks for joining the newsletter!
# => nil
``````

Setting default hash values can be tricky. The default value has to be either `nil` or `{}`. Use `nil` to make the hash optional. Use `{}` if you want to set some defaults for values inside the hash.

``````hash :optional,
default: nil
# => {:optional=>nil}

hash :with_defaults,
default: {} do
default: true
end
``````

By default, hashes remove any keys that aren't given as nested filters. To allow all hash keys, set `strip: false`. In general we don't recommend doing this, but it's sometimes necessary.

``````hash :stuff,
strip: false
``````

String

String filters define inputs that only accept strings.

``````class StringInteraction < ActiveInteraction::Base
string :name

def execute
"Hello, #{name}!"
end
end

# ActiveInteraction::InvalidInteractionError: Name is not a valid string
StringInteraction.run!(name: 'Taylor')
# => "Hello, Taylor!"
``````

String filter strips leading and trailing whitespace by default. To disable it, set the `strip` option to `false`.

``````string :comment,
strip: false
``````

Symbol

Symbol filters define inputs that accept symbols. Strings will be converted into symbols.

``````class SymbolInteraction < ActiveInteraction::Base
symbol :method

def execute
method.to_proc
end
end

SymbolInteraction.run!(method: -> {})
# ActiveInteraction::InvalidInteractionError: Method is not a valid symbol
SymbolInteraction.run!(method: :object_id)
# => #<Proc:0x007fdc9ba94118>
``````

Dates and times

Filters that work with dates and times behave similarly. By default, they all convert strings into their expected data types using `.parse`. Blank strings will be treated as `nil`. If you give the `format` option, they will instead convert strings using `.strptime`. Note that formats won't work with `DateTime` and `Time` filters if a time zone is set.

Date

``````class DateInteraction < ActiveInteraction::Base
date :birthday

def execute
birthday + (18 * 365)
end
end

DateInteraction.run!(birthday: 'yesterday')
# ActiveInteraction::InvalidInteractionError: Birthday is not a valid date
DateInteraction.run!(birthday: Date.new(1989, 9, 1))
# => #<Date: 2007-08-28 ((2454341j,0s,0n),+0s,2299161j)>
``````
``````date :birthday,
format: '%Y-%m-%d'
``````

DateTime

``````class DateTimeInteraction < ActiveInteraction::Base
date_time :now

def execute
now.iso8601
end
end

DateTimeInteraction.run!(now: 'now')
# ActiveInteraction::InvalidInteractionError: Now is not a valid date time
DateTimeInteraction.run!(now: DateTime.now)
# => "2015-03-11T11:04:40-05:00"
``````
``````date_time :start,
format: '%Y-%m-%dT%H:%M:%S'
``````

Time

In addition to converting strings with `.parse` (or `.strptime`), time filters convert numbers with `.at`.

``````class TimeInteraction < ActiveInteraction::Base
time :epoch

def execute
Time.now - epoch
end
end

TimeInteraction.run!(epoch: 'a long, long time ago')
# ActiveInteraction::InvalidInteractionError: Epoch is not a valid time
TimeInteraction.run!(epoch: Time.new(1970))
# => 1426068362.5136619
``````
``````time :start,
format: '%Y-%m-%dT%H:%M:%S'
``````

Numbers

All numeric filters accept numeric input. They will also convert strings using the appropriate method from `Kernel` (like `.Float`). Blank strings will be treated as `nil`.

Decimal

``````class DecimalInteraction < ActiveInteraction::Base
decimal :price

def execute
price * 1.0825
end
end

DecimalInteraction.run!(price: 'one ninety-nine')
# ActiveInteraction::InvalidInteractionError: Price is not a valid decimal
DecimalInteraction.run!(price: BigDecimal(1.99, 2))
# => #<BigDecimal:7fe792a42028,'0.2165E1',18(45)>
``````

To specify the number of significant digits, use the `digits` option.

``````decimal :dollars,
digits: 2
``````

Float

``````class FloatInteraction < ActiveInteraction::Base
float :x

def execute
x**2
end
end

FloatInteraction.run!(x: 'two point one')
# ActiveInteraction::InvalidInteractionError: X is not a valid float
FloatInteraction.run!(x: 2.1)
# => 4.41
``````

Integer

``````class IntegerInteraction < ActiveInteraction::Base
integer :limit

def execute
limit.downto(0).to_a
end
end

IntegerInteraction.run!(limit: 'ten')
# ActiveInteraction::InvalidInteractionError: Limit is not a valid integer
IntegerInteraction.run!(limit: 10)
# => [10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0]
``````

When a `String` is passed into an `integer` input, the value will be coerced. A default base of `10` is used though it may be overridden with the `base` option. If a base of `0` is provided, the coercion will respect radix indicators present in the string.

``````class IntegerInteraction < ActiveInteraction::Base
integer :limit1
integer :limit2, base: 8
integer :limit3, base: 0

def execute
[limit1, limit2, limit3]
end
end

IntegerInteraction.run!(limit1: 71, limit2: 71, limit3: 71)
# => [71, 71, 71]
IntegerInteraction.run!(limit1: "071", limit2: "071", limit3: "0x71")
# => [71, 57, 113]
IntegerInteraction.run!(limit1: "08", limit2: "08", limit3: "08")
ActiveInteraction::InvalidInteractionError: Limit2 is not a valid integer, Limit3 is not a valid integer
``````

Interface

Interface filters allow you to specify an interface that the passed value must meet in order to pass. The name of the interface is used to look for a constant inside the ancestor listing for the passed value. This allows for a variety of checks depending on what's passed. Class instances are checked for an included module or an inherited ancestor class. Classes are checked for an extended module or an inherited ancestor class. Modules are checked for an extended module.

``````class InterfaceInteraction < ActiveInteraction::Base
interface :exception

def execute
exception
end
end

InterfaceInteraction.run!(exception: Exception)
# ActiveInteraction::InvalidInteractionError: Exception is not a valid interface
InterfaceInteraction.run!(exception: NameError) # a subclass of Exception
# => NameError
``````

You can use `:from` to specify a class or module. This would be the equivalent of what's above.

``````class InterfaceInteraction < ActiveInteraction::Base
interface :error,
from: Exception

def execute
error
end
end
``````

You can also create an anonymous interface on the fly by passing the `methods` option.

``````class InterfaceInteraction < ActiveInteraction::Base
interface :serializer,

def execute
input = '{ "is_json" : true }'
output = serializer.dump(object)

output
end
end

require 'json'

InterfaceInteraction.run!(serializer: Object.new)
# ActiveInteraction::InvalidInteractionError: Serializer is not a valid interface
InterfaceInteraction.run!(serializer: JSON)
# => "{\"is_json\":true}"
``````

Object

Object filters allow you to require an instance of a particular class or one of its subclasses.

``````class Cow
def moo
'Moo!'
end
end

class ObjectInteraction < ActiveInteraction::Base
object :cow

def execute
cow.moo
end
end

ObjectInteraction.run!(cow: Object.new)
# ActiveInteraction::InvalidInteractionError: Cow is not a valid object
ObjectInteraction.run!(cow: Cow.new)
# => "Moo!"
``````

The class name is automatically determined by the filter name. If your filter name is different than your class name, use the `class` option. It can be either the class, a string, or a symbol.

``````object :dolly1,
class: Sheep
object :dolly2,
class: 'Sheep'
object :dolly3,
class: :Sheep
``````

If you have value objects or you would like to build one object from another, you can use the `converter` option. It is only called if the value provided is not an instance of the class or one of its subclasses. The `converter` option accepts a symbol that specifies a class method on the object class or a proc. Both will be passed the value and any errors thrown inside the converter will cause the value to be considered invalid. Any returned value that is not the correct class will also be treated as invalid. Any `default` that is not an instance of the class or subclass and is not `nil` will also be converted.

``````class ObjectInteraction < ActiveInteraction::Base
converter: :new

def execute
end
end

# ActiveInteraction::InvalidInteractionError: Ip address is not a valid object
``````

Record

Record filters allow you to require an instance of a particular class (or one of its subclasses) or a value that can be used to locate an instance of the object. If the value does not match, it will call `find` on the class of the record. This is particularly useful when working with ActiveRecord objects. Like an object filter, the class is derived from the name passed but can be specified with the `class` option. Any `default` that is not an instance of the class or subclass and is not `nil` will also be found. Blank strings passed in will be treated as `nil`.

``````class RecordInteraction < ActiveInteraction::Base
record :encoding

def execute
encoding
end
end

> RecordInteraction.run!(encoding: Encoding::US_ASCII)
=> #<Encoding:US-ASCII>

> RecordInteraction.run!(encoding: 'ascii')
=> #<Encoding:US-ASCII>
``````

A different method can be specified by providing a symbol to the `finder` option.

Rails

ActiveInteraction plays nicely with Rails. You can use interactions to handle your business logic instead of models or controllers. To see how it all works, let's take a look at a complete example of a controller with the typical resourceful actions.

Setup

We recommend putting your interactions in `app/interactions`. It's also very helpful to group them by model. That way you can look in `app/interactions/accounts` for all the ways you can interact with accounts.

``````- app/
- controllers/
- accounts_controller.rb
- interactions/
- accounts/
- create_account.rb
- destroy_account.rb
- find_account.rb
- list_accounts.rb
- update_account.rb
- models/
- account.rb
- views/
- account/
- edit.html.erb
- index.html.erb
- new.html.erb
- show.html.erb
``````

Controller

Index

``````# GET /accounts
def index
@accounts = ListAccounts.run!
end
``````

Since we're not passing any inputs to `ListAccounts`, it makes sense to use `.run!` instead of `.run`. If it failed, that would mean we probably messed up writing the interaction.

``````class ListAccounts < ActiveInteraction::Base
def execute
Account.not_deleted.order(last_name: :asc, first_name: :asc)
end
end
``````

Show

Up next is the show action. For this one we'll define a helper method to handle raising the correct errors. We have to do this because calling `.run!` would raise an `ActiveInteraction::InvalidInteractionError` instead of an `ActiveRecord::RecordNotFound`. That means Rails would render a 500 instead of a 404.

``````# GET /accounts/:id
def show
@account = find_account!
end

private

def find_account!
outcome = FindAccount.run(params)

if outcome.valid?
outcome.result
else
fail ActiveRecord::RecordNotFound, outcome.errors.full_messages.to_sentence
end
end
``````

This probably looks a little different than you're used to. Rails commonly handles this with a `before_filter` that sets the `@account` instance variable. Why is all this interaction code better? Two reasons: One, you can reuse the `FindAccount` interaction in other places, like your API controller or a Resque task. And two, if you want to change how accounts are found, you only have to change one place.

Inside the interaction, we could use `#find` instead of `#find_by_id`. That way we wouldn't need the `#find_account!` helper method in the controller because the error would bubble all the way up. However, you should try to avoid raising errors from interactions. If you do, you'll have to deal with raised exceptions as well as the validity of the outcome.

``````class FindAccount < ActiveInteraction::Base
integer :id

def execute
account = Account.not_deleted.find_by_id(id)

if account
account
else
errors.add(:id, 'does not exist')
end
end
end
``````

Note that it's perfectly fine to add errors during execution. Not all errors have to come from checking or validation.

New

The new action will be a little different than the ones we've looked at so far. Instead of calling `.run` or `.run!`, it's going to initialize a new interaction. This is possible because interactions behave like ActiveModels.

``````# GET /accounts/new
def new
@account = CreateAccount.new
end
``````

Since interactions behave like ActiveModels, we can use ActiveModel validations with them. We'll use validations here to make sure that the first and last names are not blank. The validations section goes into more detail about this.

``````class CreateAccount < ActiveInteraction::Base
string :first_name, :last_name

validates :first_name, :last_name,
presence: true

def to_model
Account.new
end

def execute
account = Account.new(inputs)

unless account.save
errors.merge!(account.errors)
end

account
end
end
``````

We used a couple of advanced features here. The `#to_model` method helps determine the correct form to use in the view. Check out the section on forms for more about that. Inside `#execute`, we merge errors. This is a convenient way to move errors from one object to another. Read more about it in the errors section.

Create

The create action has a lot in common with the new action. Both of them use the `CreateAccount` interaction. And if creating the account fails, this action falls back to rendering the new action.

``````# POST /accounts
def create
outcome = CreateAccount.run(params.fetch(:account, {}))

if outcome.valid?
redirect_to(outcome.result)
else
@account = outcome
render(:new)
end
end
``````

Note that we have to pass a hash to `.run`. Passing `nil` is an error.

Since we're using an interaction, we don't need strong parameters. The interaction will ignore any inputs that weren't defined by filters. So you can forget about `params.require` and `params.permit` because interactions handle that for you.

Destroy

The destroy action will reuse the `#find_account!` helper method we wrote earlier.

``````# DELETE /accounts/:id
def destroy
DestroyAccount.run!(account: find_account!)
redirect_to(accounts_url)
end
``````

In this simple example, the destroy interaction doesn't do much. It's not clear that you gain anything by putting it in an interaction. But in the future, when you need to do more than `account.destroy`, you'll only have to update one spot.

``````class DestroyAccount < ActiveInteraction::Base
object :account

def execute
account.destroy
end
end
``````

Edit

Just like the destroy action, editing uses the `#find_account!` helper. Then it creates a new interaction instance to use as a form object.

``````# GET /accounts/:id/edit
def edit
account = find_account!
@account = UpdateAccount.new(
account: account,
first_name: account.first_name,
last_name: account.last_name)
end
``````

The interaction that updates accounts is more complicated than the others. It requires an account to update, but the other inputs are optional. If they're missing, it'll ignore those attributes. If they're present, it'll update them.

``````class UpdateAccount < ActiveInteraction::Base
object :account

string :first_name, :last_name,
default: nil

validates :first_name,
presence: true,
unless: -> { first_name.nil? }
validates :last_name,
presence: true,
unless: -> { last_name.nil? }

def execute
account.first_name = first_name if first_name.present?
account.last_name = last_name if last_name.present?

unless account.save
errors.merge!(account.errors)
end

account
end
end
``````

Update

Hopefully you've gotten the hang of this by now. We'll use `#find_account!` to get the account. Then we'll build up the inputs for `UpdateAccount`. Then we'll run the interaction and either redirect to the updated account or back to the edit page.

``````# PUT /accounts/:id
def update
inputs = { account: find_account! }.reverse_merge(params[:account])
outcome = UpdateAccount.run(inputs)

if outcome.valid?
redirect_to(outcome.result)
else
@account = outcome
render(:edit)
end
end
``````

Callbacks

ActiveSupport::Callbacks provides a powerful framework for defining callbacks. ActiveInteraction uses that framework to allow hooking into various parts of an interaction's lifecycle.

``````class Increment < ActiveInteraction::Base
set_callback :filter, :before, -> { puts 'before filter' }

integer :x

set_callback :validate, :after, -> { puts 'after validate' }

validates :x,
numericality: { greater_than_or_equal_to: 0 }

set_callback :execute, :around, lambda { |_interaction, block|
puts '>>>'
block.call
puts '<<<'
}

def execute
puts 'executing'
x + 1
end
end

Increment.run!(x: 1)
# before filter
# after validate
# >>>
# executing
# <<<
# => 2
``````

In order, the available callbacks are `filter`, `validate`, and `execute`. You can set `before`, `after`, or `around` on any of them.

Composition

You can run interactions from within other interactions with `#compose`. If the interaction is successful, it'll return the result (just like if you had called it with `.run!`). If something went wrong, execution will halt immediately and the errors will be moved onto the caller.

``````class Add < ActiveInteraction::Base
integer :x, :y

def execute
x + y
end
end

class AddThree < ActiveInteraction::Base
integer :x

def execute
compose(Add, x: x, y: 3)
end
end

# => 8
``````

To bring in filters from another interaction, use `.import_filters`. Combined with `inputs`, delegating to another interaction is a piece of cake.

``````class AddAndDouble < ActiveInteraction::Base

def execute
compose(Add, inputs) * 2
end
end
``````

Note that errors in composed interactions have a few tricky cases. See the errors section for more information about them.

Defaults

The default value for an input can take on many different forms. Setting the default to `nil` makes the input optional. Setting it to some value makes that the default value for that input. Setting it to a lambda will lazily set the default value for that input. That means the value will be computed when the interaction is run, as opposed to when it is defined.

Lambda defaults are evaluated in the context of the interaction, so you can use the values of other inputs in them.

``````# This input is optional.
time :a, default: nil
# This input defaults to `Time.at(123)`.
time :b, default: Time.at(123)
# This input lazily defaults to `Time.now`.
time :c, default: -> { Time.now }
# This input defaults to the value of `c` plus 10 seconds.
time :d, default: -> { c + 10 }
``````

Descriptions

Use the `desc` option to provide human-readable descriptions of filters. You should prefer these to comments because they can be used to generate documentation. The interaction class has a `.filters` method that returns a hash of filters. Each filter has a `#desc` method that returns the description.

``````class Descriptive < ActiveInteraction::Base
string :first_name,
desc: 'your first name'
string :last_name,
desc: 'your last name'
end

Descriptive.filters.each do |name, filter|
puts "#{name}: #{filter.desc}"
end
# first_name: your first name
# last_name: your last name
``````

Errors

ActiveInteraction provides detailed errors for easier introspection and testing of errors. Detailed errors improve on regular errors by adding a symbol that represents the type of error that has occurred. Let's look at an example where an item is purchased using a credit card.

``````class BuyItem < ActiveInteraction::Base
object :credit_card, :item
hash :options do
end

def execute
order = credit_card.purchase(item)
notify(credit_card.account)
order
end

private def notify(account)
# ...
end
end
``````

Having missing or invalid inputs causes the interaction to fail and return errors.

``````outcome = BuyItem.run(item: 'Thing', options: { gift_wrapped: 'yes' })
outcome.errors.messages
# => {:credit_card=>["is required"], :item=>["is not a valid object"], :"options.gift_wrapped"=>["is not a valid boolean"]}
``````

Determining the type of error based on the string is difficult if not impossible. Calling `#details` instead of `#messages` on `errors` gives you the same list of errors with a testable label representing the error.

``````outcome.errors.details
# => {:credit_card=>[{:error=>:missing}], :item=>[{:error=>:invalid_type, :type=>"object"}], :"options.gift_wrapped"=>[{:error=>:invalid_type, :type=>"boolean"}]}
``````

Detailed errors can also be manually added during the execute call by passing a symbol to `#add` instead of a string.

``````def execute
end
``````

ActiveInteraction also supports merging errors. This is useful if you want to delegate validation to some other object. For example, if you have an interaction that updates a record, you might want that record to validate itself. By using the `#merge!` helper on `errors`, you can do exactly that.

``````class UpdateThing < ActiveInteraction::Base
object :thing

def execute
unless thing.save
errors.merge!(thing.errors)
end

thing
end
end
``````

When a composed interaction fails, its errors are merged onto the caller. This generally produces good error messages, but there are a few cases to look out for.

``````class Inner < ActiveInteraction::Base
boolean :x, :y
end

class Outer < ActiveInteraction::Base
string :x
boolean :z, default: nil

def execute
compose(Inner, x: x, y: z)
end
end

outcome = Outer.run(x: 'yes')
outcome.errors.details
# => { :x    => [{ :error => :invalid_type, :type => "boolean" }],
#      :base => [{ :error => "Y is required" }] }
outcome.errors.full_messages.join(' and ')
# => "X is not a valid boolean and Y is required"
``````

Since both interactions have an input called `x`, the inner error for that input is moved to the `x` error on the outer interaction. This results in a misleading error that claims the input `x` is not a valid boolean even though it's a string on the outer interaction.

Since only the inner interaction has an input called `y`, the inner error for that input is moved to the `base` error on the outer interaction. This results in a confusing error that claims the input `y` is required even though it's not present on the outer interaction.

Forms

The outcome returned by `.run` can be used in forms as though it were an ActiveModel object. You can also create a form object by calling `.new` on the interaction.

Given an application with an `Account` model we'll create a new `Account` using the `CreateAccount` interaction.

``````# GET /accounts/new
def new
@account = CreateAccount.new
end

# POST /accounts
def create
outcome = CreateAccount.run(params.fetch(:account, {}))

if outcome.valid?
redirect_to(outcome.result)
else
@account = outcome
render(:new)
end
end
``````

The form used to create a new `Account` has slightly more information on the `form_for` call than you might expect.

``````<%= form_for @account, as: :account, url: accounts_path do |f| %>
<%= f.text_field :first_name %>
<%= f.text_field :last_name %>
<%= f.submit 'Create' %>
<% end %>
``````

This is necessary because we want the form to act like it is creating a new `Account`. Defining `to_model` on the `CreateAccount` interaction tells the form to treat our interaction like an `Account`.

``````class CreateAccount < ActiveInteraction::Base
# ...

def to_model
Account.new
end
end
``````

Now our `form_for` call knows how to generate the correct URL and param name (i.e. `params[:account]`).

``````# app/views/accounts/new.html.erb
<%= form_for @account do |f| %>
<%# ... %>
<% end %>
``````

If you have an interaction that updates an `Account`, you can define `to_model` to return the object you're updating.

``````class UpdateAccount < ActiveInteraction::Base
# ...

object :account

def to_model
account
end
end
``````

ActiveInteraction also supports formtastic and simple_form. The filters used to define the inputs on your interaction will relay type information to these gems. As a result, form fields will automatically use the appropriate input type.

Shared input options

It can be convenient to apply the same options to a bunch of inputs. One common use case is making many inputs optional. Instead of setting `default: nil` on each one of them, you can use `with_options` to reduce duplication.

``````with_options default: nil do
date :birthday
string :name
boolean :wants_cake
end
``````

Optional inputs

Optional inputs can be defined by using the `:default` option as described in the filters section. Within the interaction, provided and default values are merged to create `inputs`. There are times where it is useful to know whether a value was passed to `run` or the result of a filter default. In particular, it is useful when `nil` is an acceptable value. For example, you may optionally track your users' birthdays. You can use the `inputs.given?` predicate to see if an input was even passed to `run`. With `inputs.given?` you can also check the input of a hash or array filter by passing a series of keys or indexes to check.

``````class UpdateUser < ActiveInteraction::Base
object :user
date :birthday,
default: nil

def execute
user.birthday = birthday if inputs.given?(:birthday)
errors.merge!(user.errors) unless user.save
user
end
end
``````

Now you have a few options. If you don't want to update their birthday, leave it out of the hash. If you want to remove their birthday, set `birthday: nil`. And if you want to update it, pass in the new value as usual.

``````user = User.find(...)

# Don't update their birthday.
UpdateUser.run!(user: user)

# Remove their birthday.
UpdateUser.run!(user: user, birthday: nil)

# Update their birthday.
UpdateUser.run!(user: user, birthday: Date.new(2000, 1, 2))
``````

Translations

ActiveInteraction is i18n aware out of the box! All you have to do is add translations to your project. In Rails, these typically go into `config/locales`. For example, let's say that for some reason you want to print everything out backwards. Simply add translations for ActiveInteraction to your `hsilgne` locale.

``````# config/locales/hsilgne.yml
hsilgne:
active_interaction:
types:
array: yarra
boolean: naeloob
decimal: lamiced
file: elif
float: taolf
hash: hsah
integer: regetni
interface: ecafretni
object: tcejbo
string: gnirts
symbol: lobmys
time: emit
errors:
messages:
invalid: dilavni si
invalid_type: '%{type} dilav a ton si'
missing: deriuqer si
``````

Then set your locale and run interactions like normal.

``````class I18nInteraction < ActiveInteraction::Base
string :name
end

I18nInteraction.run(name: false).errors.messages[:name]
# => ["is not a valid string"]

I18n.locale = :hsilgne
I18nInteraction.run(name: false).errors.messages[:name]
# => ["gnirts dilav a ton si"]
``````

Everything else works like an `activerecord` entry. For example, to rename an attribute you can use `attributes`.

Here we'll rename the `num` attribute on an interaction named `product`:

``````en:
active_interaction:
attributes:
product:
num: 'Number'
``````

Credits

ActiveInteraction is brought to you by Aaron Lasseigne. Along with Aaron, Taylor Fausak helped create and maintain ActiveInteraction but has since moved on.

If you want to contribute to ActiveInteraction, please read our contribution guidelines. A complete list of contributors is available on GitHub.

Author: AaronLasseigne
Source code: https://github.com/AaronLasseigne/active_interaction

1659511140

Roadie: Making HTML Emails Comfortable for The Ruby Rockstars

Making HTML emails comfortable for the Ruby rockstars

Roadie tries to make sending HTML emails a little less painful by inlining stylesheets and rewriting relative URLs for you inside your emails.

How does it work?

Email clients have bad support for stylesheets, and some of them blocks stylesheets from downloading. The easiest way to handle this is to work with inline styles (`style="..."`), but that is error prone and hard to work with as you cannot use classes and/or reuse styling over your HTML.

This gem makes this easier by automatically inlining stylesheets into the document. You give Roadie your CSS, or let it find it by itself from the `<link>` and `<style>` tags in the markup, and it will go through all of the selectors assigning the styles to the matching elements. Careful attention has been put into selectors being applied in the correct order, so it should behave just like in the browser.

"Dynamic" selectors (`:hover`, `:visited`, `:focus`, etc.), or selectors not understood by Nokogiri will be inlined into a single `<style>` element for those email clients that support it. This changes specificity a great deal for these rules, so it might not work 100% out of the box. (See more about this below)

Roadie also rewrites all relative URLs in the email to an absolute counterpart, making images you insert and those referenced in your stylesheets work. No more headaches about how to write the stylesheets while still having them work with emails from your acceptance environments. You can disable this on specific elements using a `data-roadie-ignore` marker.

Features

• Writes CSS styles inline.
• Respects `!important` styles.
• Does not overwrite styles already present in the `style` attribute of tags.
• Supports the same CSS selectors as Nokogiri; use CSS3 selectors in your emails!
• Keeps `:hover`, `@media { ... }` and friends around in a separate `<style>` element.
• Makes image urls absolute.
• Hostname and port configurable on a per-environment basis.
• Can be disabled on individual elements.
• Makes link `href`s and `img` `src`s absolute.
• Automatically adds proper HTML skeleton when missing; you don't have to create a layout for emails.
• Also supports HTML fragments / partial documents, where layout is not added.
• Allows you to inject stylesheets in a number of ways, at runtime.
• Removes `data-roadie-ignore` markers before finishing the HTML.

Install & Usage

Add this gem to your Gemfile as recommended by Rubygems and run `bundle install`.

``````gem 'roadie', '~> 4.0'
``````

Your document instance can be configured with several options:

• `url_options` - Dictates how absolute URLs should be built.
• `keep_uninlinable_css` - Set to false to skip CSS that cannot be inlined.
• `merge_media_queries` - Set to false to not group media queries. Some users might prefer to not group rules within media queries because it will result in rules getting reordered. e.g.
``````@media(max-width: 600px) { .col-6 { display: block; } }
@media(max-width: 400px) { .col-12 { display: inline-block; } }
@media(max-width: 600px) { .col-12 { display: block; } }``````
• will become
``````@media(max-width: 600px) { .col-6 { display: block; } .col-12 { display: block; } }
@media(max-width: 400px) { .col-12 { display: inline-block; } }``````
• `asset_providers` - A list of asset providers that are invoked when CSS files are referenced. See below.
• `external_asset_providers` - A list of asset providers that are invoked when absolute CSS URLs are referenced. See below.
• `before_transformation` - A callback run before transformation starts.
• `after_transformation` - A callback run after transformation is completed.

Making URLs absolute

In order to make URLs absolute you need to first configure the URL options of the document.

``````html = '... <a href="/about-us">Read more!</a> ...'
document = Roadie::Document.new html
document.url_options = {host: "myapp.com", protocol: "https"}
document.transform
# => "... <a href=\"https://myapp.com/about-us\">Read more!</a> ..."
``````

The following URLs will be rewritten for you:

• `a[href]` (HTML)
• `img[src]` (HTML)
• `url()` (CSS)

You can disable individual elements by adding an `data-roadie-ignore` marker on them. CSS will still be inlined on those elements, but URLs will not be rewritten.

``````<a href="|UNSUBSCRIBE_URL|" data-roadie-ignore>Unsubscribe</a>
``````

Referenced stylesheets

By default, `style` and `link` elements in the email document's `head` are processed along with the stylesheets and removed from the `head`.

You can set a special `data-roadie-ignore` attribute on `style` and `link` tags that you want to ignore (the attribute will be removed, however). This is the place to put things like `:hover` selectors that you want to have for email clients allowing them.

Style and link elements with `media="print"` are also ignored.

``````<head>
<link rel="stylesheet" type="text/css" href="/assets/emails/rock.css">         <!-- Will be inlined with normal providers -->
<link rel="stylesheet" type="text/css" href="http://www.metal.org/metal.css">  <!-- Will be inlined with external providers, *IF* specified; otherwise ignored. -->
<link rel="stylesheet" type="text/css" href="/assets/jazz.css" media="print">  <!-- Will NOT be inlined; print style -->
<link rel="stylesheet" type="text/css" href="/ambient.css" data-roadie-ignore> <!-- Will NOT be inlined; ignored -->
<style></style>                    <!-- Will be inlined -->
<style data-roadie-ignore></style> <!-- Will NOT be inlined; ignored -->
``````

Roadie will use the given asset providers to look for the actual CSS that is referenced. If you don't change the default, it will use the `Roadie::FilesystemProvider` which looks for stylesheets on the filesystem, relative to the current working directory.

Example:

``````# /home/user/foo/stylesheets/primary.css
body { color: green; }

# /home/user/foo/script.rb
html = <<-HTML
<html>
<link rel="stylesheet" type="text/css" href="/stylesheets/primary.css">
<body>
</body>
</html>
HTML

Dir.pwd # => "/home/user/foo"
document = Roadie::Document.new html
document.transform # =>
# <!DOCTYPE html>
# <html>
#   <body style="color:green;"></body>
# </html>
``````

If a referenced stylesheet cannot be found, the `#transform` method will raise an `Roadie::CssNotFound` error. If you instead want to ignore missing stylesheets, you can use the `NullProvider`.

Configuring providers

You can write your own providers if you need very specific behavior for your app, or you can use the built-in providers. Providers come in two groups: normal and external. Normal providers handle paths without host information (`/style/foo.css`) while external providers handle URLs with host information (`//example.com/foo.css`, `localhost:3001/bar.css`, and so on).

The default configuration is to not have any external providers configured, which will cause those referenced stylesheets to be ignored. Adding one or more providers for external assets causes all of them to be searched and inlined, so if you only want this to happen to specific stylesheets you need to add ignore markers to every other styleshheet (see above).

Included providers:

• `FilesystemProvider` – Looks for files on the filesystem, relative to the given directory unless otherwise specified.
• `ProviderList` – Wraps a list of other providers and searches them in order. The `asset_providers` setting is an instance of this. It behaves a lot like an array, so you can push, pop, shift and unshift to it.
• `NullProvider` – Does not actually provide anything, it always finds empty stylesheets. Use this in tests or if you want to ignore stylesheets that cannot be found by your other providers (or if you want to force the other providers to never run).
• `NetHttpProvider` – Downloads stylesheets using `Net::HTTP`. Can be given a whitelist of hosts to download from.
• `CachedProvider` – Wraps another provider (or `ProviderList`) and caches responses inside the provided cache store.
• `PathRewriterProvider` – Rewrites the passed path and then passes it on to another provider (or `ProviderList`).

If you want to search several locations on the filesystem, you can declare that:

``````document.asset_providers = [
]
``````

`NullProvider`

If you want to ignore stylesheets that cannot be found instead of crashing, push the `NullProvider` to the end:

``````# Don't crash on missing assets

``````

Note: This will cause the referenced stylesheet to be removed from the source code, so email client will never see it either.

`NetHttpProvider`

The `NetHttpProvider` will download the URLs that is is given using Ruby's standard `Net::HTTP` library.

You can give it a whitelist of hosts that downloads are allowed from:

``````document.external_asset_providers << Roadie::NetHttpProvider.new(
whitelist: ["myapp.com", "assets.myapp.com", "cdn.cdnnetwork.co.jp"],
)
document.external_asset_providers << Roadie::NetHttpProvider.new # Allows every host
``````

`CachedProvider`

You might want to cache providers from working several times. If you are sending several emails quickly from the same process, this might also save a lot of time on parsing the stylesheets if you use in-memory storage such as a hash.

You can wrap any other kind of providers with it, even a `ProviderList`:

``````document.external_asset_providers = Roadie::CachedProvider.new(document.external_asset_providers, my_cache)
``````

If you don't pass a cache backend, it will use a normal `Hash`. The cache store must follow this protocol:

``````my_cache["key"] = some_stylesheet_instance # => #<Roadie::Stylesheet instance>
my_cache["key"]                            # => #<Roadie::Stylesheet instance>
my_cache["missing"]                        # => nil
``````

Warning: The default `Hash` store will never be cleared, so make sure you don't allow the number of unique asset paths to grow too large in a single run. This is especially important if you run Roadie in a daemon that accepts arbritary documents, and/or if you use hash digests in your filenames. Making a new instance of `CachedProvider` will use a new `Hash` instance.

You can implement your own custom cache store by implementing the `[]` and `[]=` methods.

``````class MyRoadieMemcacheStore
def initialize(memcache)
@memcache = memcache
end

def [](path)
if css
name = memcache.read("assets/#{path}/name") || "cached #{path}"
end
end

def []=(path, stylesheet)
memcache.write("assets/#{path}/css", stylesheet.to_s)
memcache.write("assets/#{path}/name", stylesheet.name)
stylesheet # You need to return the set Stylesheet
end
end

document.external_asset_providers,
)
``````

If you are using Rspec, you can test your implementation by using the shared examples for the "roadie cache store" role:

``````require "roadie/rspec"

let(:memcache_client) { MemcacheClient.instance }
subject { MyRoadieMemcacheStore.new(memcache_client) }

it_behaves_like "roadie cache store" do
before { memcache_client.clear }
end
end
``````

`PathRewriterProvider`

With this provider, you can rewrite the paths that are searched in order to more easily support another provider. Examples could include rewriting absolute URLs into something that can be found on the filesystem, or to access internal hosts instead of external ones.

``````filesystem = Roadie::FilesystemProvider.new("assets")
document.asset_providers << Roadie::PathRewriterProvider.new(filesystem) do |path|
path.sub('stylesheets', 'css').downcase
end

document.external_asset_providers = Roadie::PathRewriterProvider.new(filesystem) do |url|
if url =~ /myapp\.com/
URI.parse(url).path.sub(%r{^/assets}, '')
else
url
end
end
``````

You can also wrap a list, for example to implement `external_asset_providers` by composing the normal `asset_providers`:

``````document.external_asset_providers =
URI.parse(url).path
end
``````

Writing your own provider

Writing your own provider is also easy. You need to provide:

• `#find_stylesheet(name)`, returning either a `Roadie::Stylesheet` or `nil`.
• `#find_stylesheet!(name)`, returning either a `Roadie::Stylesheet` or raising `Roadie::CssNotFound`.
``````class UserAssetsProvider
def initialize(user_collection)
@user_collection = user_collection
end

def find_stylesheet(name)
if name =~ %r{^/users/(\d+)\.css\$}
user = @user_collection.find_user(\$1)
Roadie::Stylesheet.new("user #{user.id} stylesheet", user.stylesheet)
end
end

def find_stylesheet!(name)
find_stylesheet(name) or
css_name: name, message: "does not match a user stylesheet", provider: self
)
end

# Instead of implementing #find_stylesheet!, you could also:
# That will give you a default implementation without any error message. If
# you have multiple error cases, it's recommended that you implement
# #find_stylesheet! without #find_stylesheet and raise with an explanatory
# error message.
end

# Try to look for a user stylesheet first, then fall back to normal filesystem lookup.
document.asset_providers = [
UserAssetsProvider.new(app),
]
``````

You can test for compliance by using the built-in RSpec examples:

``````require 'spec_helper'

describe MyOwnProvider do
# Will use the default `subject` (MyOwnProvider.new)
it_behaves_like "roadie asset provider", valid_name: "found.css", invalid_name: "does_not_exist.css"

# Extra setup just for these tests:
it_behaves_like "roadie asset provider", valid_name: "found.css", invalid_name: "does_not_exist.css" do
subject { MyOwnProvider.new(...) }
before { stub_dependencies }
end
end
``````

Keeping CSS that is impossible to inline

Some CSS is impossible to inline properly. `:hover` and `::after` comes to mind. Roadie tries its best to keep these around by injecting them inside a new `<style>` element in the `<head>` (or at the beginning of the partial if transforming a partial document).

The problem here is that Roadie cannot possible adjust the specificity for you, so they will not apply the same way as they did before the styles were inlined.

Another caveat is that a lot of email clients does not support this (which is the entire point of inlining in the first place), so don't put anything important in here. Always handle the case of these selectors not being part of the email.

Specificity problems

Inlined styles will have much higher specificity than styles in a `<style>`. Here's an example:

``````<style>p:hover { color: blue; }</style>
<p style="color: green;">Hello world</p>
``````

When hovering over this `<p>`, the color will not change as the `color: green` rule takes precedence. You can get it to work by adding `!important` to the `:hover` rule.

It would be foolish to try to automatically inject `!important` on every rule automatically, so this is a manual process.

Turning it off

If you'd rather skip this and have the styles not possible to inline disappear, you can turn off this feature by setting the `keep_uninlinable_css` option to false.

``````document.keep_uninlinable_css = false
``````

Callbacks

Callbacks allow you to do custom work on documents before they are transformed. The Nokogiri document tree is passed to the callable along with the `Roadie::Document` instance:

``````class TrackNewsletterLinks
def call(dom, document)
end

divider = (link['href'] =~ /?/ ? '&' : '?')
end
end

document.before_transformation = ->(dom, document) {
logger.debug "Inlining document with title #{dom.at_css('head > title').try(:text)}"
}
``````

XHTML vs HTML

You can configure the underlying HTML/XML engine to output XHTML or HTML (which is the default). One usecase for this is that `{` tokens usually gets escaped to `&#123;`, which would be a problem if you then pass the resulting HTML on to some other templating engine that uses those tokens (like Handlebars or Mustache).

``````document.mode = :xhtml
``````

This will also affect the emitted `<!DOCTYPE>` if transforming a full document. Partial documents does not have a `<!DOCTYPE>`.

Build Status

Tested with Github CI using:

• MRI 2.6
• MRI 2.7
• MRI 3.0
• MRI 3.1

Let me know if you want any other runtime supported officially.

Versioning

This project follows Semantic Versioning and has been since version 1.0.0.

FAQ

Why is my markup changed in subtle ways?

Roadie uses Nokogiri to parse and regenerate the HTML of your email, which means that some unintentional changes might show up.

One example would be that Nokogiri might remove your `&nbsp;`s in some cases.

Another example is Nokogiri's lack of HTML5 support, so certain new element might have spaces removed. I recommend you don't use HTML5 in emails anyway because of bad email client support (that includes web mail!).

I'm getting segmentation faults (or other C-like problems)! What should I do?

Roadie uses Nokogiri to parse the HTML of your email, so any C-like problems like segfaults are likely in that end. The best way to fix this is to first upgrade libxml2 on your system and then reinstall Nokogiri. Instructions on how to do this on most platforms, see Nokogiri's official install guide.

What happened to my `@keyframes`?

The CSS Parser used in Roadie does not handle keyframes. I don't think any email clients do either, but if you want to keep on trying you can add them manually to a `<style>` element (or a separate referenced stylesheet) and tell Roadie not to touch them.

My `@media` queries are reordered, how can I fix this?

Different `@media` query blocks with the same conditions are merged by default, which will change the order in some cases. You can disable this by setting `merge_media_queries` to `false`. (See Install & Usage section above).

How do I get rid of the `<body>` elements that are added?

It sounds like you want to transform a partial document. Maybe you are building partials or template fragments to later place in other documents. Use `Document#transform_partial` instead of `Document#transform` in order to treat the HTML as a partial document.

Can I skip URL rewriting on a specific element?

If you add the `data-roadie-ignore` attribute on an element, URL rewriting will not be performed on that element. This could be really useful for you if you intend to send the email through some other rendering pipeline that replaces some placeholders/variables.

``````<a href="/about-us">About us</a>
``````

Note that this will not skip CSS inlining on the element; it will still get the correct styles applied.

What should I do about "Invalid URL" errors?

If the URL is invalid on purpose, see Can I skip URL rewriting on a specific element? above. Otherwise, you can try to parse it yourself using Ruby's `URI` class and see if you can figure it out.

``````require "uri"
URI.parse("https://example.com/best image.jpg") # raises
URI.parse("https://example.com/best%20image.jpg") # Works!
``````

Running specs

``````bundle install
rake
``````

Security

Roadie is set up with the assumption that all CSS and HTML passing through it is under your control. It is not recommended to run arbritary HTML with the default settings.

Care has been given to try to secure all file system accesses, but it is never guaranteed that someone cannot access something they should not be able to access.

In order to secure Roadie against file system access, only use your own asset providers that you yourself can secure against your particular environment.

If you have found any security vulnerability, please email me at `magnus.bergmark+security@gmail.com` to disclose it. For very sensitive issues, please use my public GPG key. You can also encrypt your message with my public key and open an issue if you do not want to email me directly. Thank you.

History and contributors

This gem was previously tied to Rails. It is now framework-agnostic and supports any type of HTML documents. If you want to use it with Rails, check out roadie-rails.

Major contributors to Roadie:

You can see all contributors on GitHub.