From summarizing distributions to pooling operators and measuring the goodness of fit, averaging plays a unique and universally recognized role in the field of machine learning. In the talk we present Generalized Average (GA) - a continuous and fully differentiable average operator that allows for flexible interpolation between min, max and three different types of averages: arithmetic, geometric and harmonic. We share the results of two lines of experiments: (1) using GA in hyperparameter tuning for false-positive-averse cases (e.g. fraud detection) and (2) using GA as a pooling operator in Graph Attention Networks to improve the model’s flexibility. Finally, we present an open-source Python package with our implementation of GA. The talk is addressed to machine learning practitioners, who are interested in enriching their toolbox.
