After version 2.4, the Google Brain team has now released the upgraded version of TensorFlow, version 2.5.0. The latest version comes with several new and improved features. TensorFlow 2.5 now supports Python 3.9, and TensorFlow pip packages are now built with CUDA11.2 and cuDNN 8.1.0. In this article, we discuss the major updates and features of TensorFlow 2.5.0.
tf.data
:
tf.data
service now supports strict round-robin reads, which is useful for synchronous training workloads where example sizes vary. With strict round robin reads, users can guarantee that consumers get similar-sized examples in the same step.compression=None
to tf.data.experimental.service.distribute(...)
.tf.data.Dataset.batch()
now supports num_parallel_calls
and deterministic
arguments. num_parallel_calls
is used to indicate that multiple input batches should be computed in parallel. With num_parallel_calls
set, deterministic
is used to indicate that outputs can be obtained in the non-deterministic order.tf.data.Dataset.options()
are no longer mutable.map
. The debug mode can be enabled through tf.data.experimental.enable_debug_mode()
.tf.lite
experimental_new_quantizer
in tf.lite.TFLiteConverter to False to disable this changetf.keras
tf.keras.metrics.AUC
now support logit predictions.Model.fit
, tf.keras.utils.experimental.DatasetCreator
, which takes a callable, dataset_fn
. DatasetCreator
is intended to work across all tf.distribute
strategies, and is the only input type supported for Parameter Server strategy.tf.distribute
tf.distribute.experimental.ParameterServerStrategy
now supports training with Keras Model.fit
when used with DatasetCreator
.tf.random.Generator
under tf.distribute.Strategy
scopes is now allowed (except for tf.distribute.experimental.CentralStorageStrategy
and tf.distribute.experimental.ParameterServerStrategy
). Different replicas will get different random-number streams.profile_data_directory
to EmbeddingConfigSpec
in _tpu_estimator_embedding.py
. This allows embedding lookup statistics gathered at runtime to be used in embedding layer partitioning decisions.TF_ENABLE_ONEDNN_OPTS=1
.TF_CPP_MIN_VLOG_LEVEL
environment variable has been renamed to to TF_CPP_MAX_VLOG_LEVEL
which correctly describes its effect.tf.keras
:
StringLookup
added output_mode
, sparse
, and pad_to_max_tokens
arguments with same semantics as TextVectorization
.IntegerLookup
added output_mode
, sparse
, and pad_to_max_tokens
arguments with same semantics as TextVectorization
. Renamed max_values
, oov_value
and mask_value
to max_tokens
, oov_token
and mask_token
to align with StringLookup
and TextVectorization
.TextVectorization
default for pad_to_max_tokens
switched to False.CategoryEncoding
no longer supports adapt
, IntegerLookup
now supports equivalent functionality. max_tokens
argument renamed to num_tokens
.Discretization
added num_bins
argument for learning bins boundaries through calling adapt
on a dataset. Renamed bins
argument to bin_boundaries
for specifying bins without adapt
.model.load_weights
now accepts paths to saved models.tf.TypeSpecs
.tf.keras.optimizers.schedules.CosineDecay
andtf.keras.optimizers.schedules.CosineDecayRestarts
.tf.data
:
tf.data.experimental.ExternalStatePolicy
, which can be used to control how external state should be handled during dataset serialization or iterator checkpointing.tf.data.experimental.save
to store the type specification of the dataset elements. This avoids the need for explicitly specifying the element_spec
argument of tf.data.experimental.load
when loading the previously saved dataset..element_spec
property to tf.data.DatasetSpec
to access the inner spec. This can be used to extract the structure of nestedtf.data.experimental.AutoShardingPolicy.HINT
which can be used to provide hints to tf.distribute-based auto-sharding as to where in the input pipeline to insert sharding transformations.tf.function
and GraphDef
boundaries.XLA compilation:
tf.function(experimental_compile=True)
has become a stable API, renamed tf.function(jit_compile=True)
.strategy.run
can now be annoted with jit_compile=True
.tf.distribute
:
experimental_prefetch_to_device
in tf.distribute.InputOptions
to experimental_fetch_to_device
to better reflect the purpose.tf.lite
:
tflite::Subgraph
:
tensors()
method and the non-const overload of the nodes_and_registration()
method, both of which were previously documented as temporary and to be removed.
tensors()
can be replaced by calling the existing methods tensors_size()
and tensor(int)
.nodes_and_registration
can be replaced by calling the existing methods nodes_size()
and context()
, and then calling the GetNodeAndRegistration
method in the TfLiteContext
returned by context()
.Interpreter::UseNNAPI(bool)
C++ API.
NnApiDelegate()
and related delegate configuration methods directly.16 bits quantization
Added int16x8 support for ABS, REDUCE_MAX and REDUCE_MIN operators.
Additional tests and fixes for ADD and SUB operators.
Added support for saved model’s session initializer through TFLiteConverter.from_saved_model
.
Added DEPTH_TO_SPACE support in Post training quantization.
Added dynamic range quantization support for the BatchMatMul op.
Add RFFT2D
as builtin op. (RFFT2D
also supports RFFTD
.) Currently only supports float32 input.
Add 5D support to SLICE
op.
TFLite Supports SingatureDef:
Add int8 support for ReshapeV2
.
Add experimental support for optimization with sparsity.
Add nominal support for unsigned 32-bit integer tensor types. Note that very few TFLite kernels support this type natively, so its use in mobile ML authoring is generally discouraged.
Add support for static hash tables through TFLiteConverter.from_saved_model
.
The Python TF Lite Interpreter bindings now has an option experimental_preserve_all_tensors
to aid in debugging conversion.
Quantized x86 execution defaults to Ruy GEMM library for platforms with AVX support.
Deprecate tf.compat.v1.lite.experimental.get_potentially_supported_ops
. Use tf.lite.TFLiteConverter
directly to check whether a model is convertible.
Add support to select one of three different built-in op resolvers
Enabled post training with calibrations for models that require user provided TensorFlow Lite custom op libraries via converter.target_spec._experimental_custom_op_registerers
. used in Python Interpreter API.
TF Core:
tf.cond
, tf.while_loop
, and compositions like tf.foldl
) computed with tf.GradientTape
inside a tf.function
.gradient_checker_v2.compute_gradients
to be exactly representable as a binary floating point numbers. This avoids poluting gradient approximations needlessly, which is some cases leads to false negatives in op gradient tests.tf.config.experimental.get_memory_info
, returning a dict with the current and peak memory usage. Deprecated tf.config.experimental.get_memory_usage
in favor of this new function.tf.config.experimental.enable_tensor_float_32_execution
to control Tensor-Float-32 evaluation in RNNs.tf.summary
:
tf.summary.graph
allows manual write of TensorFlow graph (tf.Graph
or tf.compat.v1.GraphDef
) as a summary. This is not a replacement for the trace-based API.Set /d2ReducedOptimizeHugeFunctions
by default for Windows builds. This provides a big compile-time speedup, and effectively raises the minimum supported MSVC version to 16.4 (current: 16.8).
TensorRT
session_config
parameter for the TF1-TRT converter TrtGraphConverter
. Previously, we issued a warning when the value of the parameter is not None.TrtGraphConverterV2
takes an object of class TrtConversionParams as a parameter. Removed three deprecated fields from this class: rewriter_config_template
, is_dynamic_op
, and max_batch_size
. Previously, we issued a warning when the value of rewriter_config_template
is not None. We issued an error when the value of is_dynamic_op
is not True. We didn’t use the value for max_batch_size
for building TensorRT engines. Add parameters use_dynamic_shape
to enable dynamic shape support. The default is to disable dynamic shape support. Add dynamic_shape_profile_strategy
for selecting a dynamic shape profile strategy. The default is profile strategy is Range
.TF XLA
MLIR_BRIDGE_ROLLOUT_SAFE_MODE_ENABLED
to tf.config.experimental.mlir_bridge_rollout
to enable a “safe” mode. This runs the MLIR bridge only when an analysis of the graph only when an analysis of the graph determines that it is safe to run.MLIR_BRIDGE_ROLLOUT_SAFE_MODE_FALLBACK_ENABLED' to
tf.config.experimental.mlir_bridge_rollout` to enable a fallback for the MLIR bridge in a “safe” mode. This runs the MLIR bridge in a FallbackEnabled mode when an analysis of the graph determines that the graph does not have unsupported features.Deterministic Op Functionality:
TF_DETERMINISTIC_OPS
is set to "true"
or "1"
(when op-determinism is expected), an attempt to run the following ops on a GPU will throw tf.errors.UnimplementedError
(with an understandable message) when data
is a floating-point type, including complex types (if supported): tf.math.segment_prod
, tf.math.segment_sum
, tf.math.unsorted_segment_mean
, tf.math.unsorted_segment_sqrt_n
, tf.math.unsorted_segment_prod
, tf.math.unsorted_segment_sum
, and therefore also tf.convert_to_tensor
when value
is of type tf.IndexedSlices
(such as in the back prop though tf.gather
into a dense embedding). See issue 39751 which this change addresses, but does not solve. This exception-throwing behavior can be disabled by setting the environment variable TF_DISABLE_SEGMENT_REDUCTION_OP_DETERMINISM_EXCEPTIONS
to "true"
or "1"
. For more information about these changes, see the description in pull request 47772.tf.sparse.sparse_dense_matmul
introduced truly random noise in the forward path for data of type tf.float32
but not for data of type tf.float64
(for which there was no GPU implementation). In this current release, GPU support for other floating-point types (tf.float16
, tf.float64
, tf.complex64
, and tf.complex128
) has been added for this op. If you were relying on the determinism of the tf.float64
CPU implementation being automatically selected because of the absence of the tf.float64
GPU implementation, you with either need to force the op to run on the CPU or use a different data type.Security
RaggedBinCount
(CVE-2021-29512)RaggedBinCount
(CVE-2021-29514)MatrixDiag*
ops (CVE-2021-29515)Conv3D
(CVE-2021-29517)CHECK
-fail in SparseCross
caused by type confusion (CVE-2021-29519)SparseCountSparseOutput
(CVE-2021-29521)Conv3DBackprop*
(CVE-2021-29520)Conv3DBackprop*
(CVE-2021-29522)CHECK
-fail in AddManySparseToTensorsMap
(CVE-2021-29523)Conv2DBackpropFilter
(CVE-2021-29524)Conv2DBackpropInput
(CVE-2021-29525)Conv2D
(CVE-2021-29526)QuantizedConv2D
(CVE-2021-29527)QuantizedMul
(CVE-2021-29528)SparseMatrixSparseCholesky
(CVE-2021-29530)CHECK
-fail in tf.raw_ops.EncodePng
(CVE-2021-29531)RaggedCross
(CVE-2021-29532)CHECK
-fail in DrawBoundingBoxes
(CVE-2021-29533)QuantizedMul
(CVE-2021-29535)CHECK
-fail in SparseConcat
(CVE-2021-29534)QuantizedResizeBilinear
(CVE-2021-29537)QuantizedReshape
(CVE-2021-29536)Conv2DBackpropFilter
(CVE-2021-29538)Conv2DBackpropFilter
(CVE-2021-29540)StringNGrams
(CVE-2021-29542)StringNGrams
(CVE-2021-29541)CHECK
-fail in QuantizeAndDequantizeV4Grad
(CVE-2021-29544)CHECK
-fail in CTCGreedyDecoder
(CVE-2021-29543)SparseTensorToCSRSparseMatrix
(CVE-2021-29545)QuantizedBiasAdd
(CVE-2021-29546)QuantizedBatchNormWithGlobalNormalization
(CVE-2021-29547)QuantizedBatchNormWithGlobalNormalization
(CVE-2021-29548)QuantizedAdd
(CVE-2021-29549)FractionalAvgPool
(CVE-2021-29550)MatrixTriangularSolve
(CVE-2021-29551)QuantizeAndDequantizeV3
(CVE-2021-29553)CHECK
-failure in UnsortedSegmentJoin
(CVE-2021-29552)DenseCountSparseOutput
(CVE-2021-29554)FusedBatchNorm
(CVE-2021-29555)SparseMatMul
(CVE-2021-29557)Reverse
(CVE-2021-29556)SparseSplit
(CVE-2021-29558)RaggedTensorToTensor
(CVE-2021-29560)CHECK
-fail in LoadAndRemapMatrix
(CVE-2021-29561)CHECK
-fail in tf.raw_ops.IRFFT
(CVE-2021-29562)CHECK
-fail in tf.raw_ops.RFFT
(CVE-2021-29563)EditDistance
(CVE-2021-29564)SparseFillEmptyRows
(CVE-2021-29565)Dilation2DBackpropInput
(CVE-2021-29566)ParameterizedTruncatedNormal
(CVE-2021-29568)SparseDenseCwiseMul
(CVE-2021-29567)MaxPoolGradWithArgmax
(CVE-2021-29570)RequantizationRange
(CVE-2021-29569)DrawBoundingBoxesV2
(CVE-2021-29571)SdcaOptimizer
(CVE-2021-29572)tf.raw_ops.ReverseSequence
(CVE-2021-29575)MaxPoolGradWithArgmax
(CVE-2021-29573)MaxPool3DGradGrad
(CVE-2021-29574)MaxPool3DGradGrad
(CVE-2021-29576)AvgPool3DGrad
(CVE-2021-29577)CHECK
-fail in FractionalMaxPoolGrad
(CVE-2021-29580)FractionalAvgPoolGrad
(CVE-2021-29578)MaxPoolGrad
(CVE-2021-29579)CTCBeamSearchDecoder
(CVE-2021-29581)tf.raw_ops.Dequantize
(CVE-2021-29582)CHECK
-fail due to integer overflow (CVE-2021-29584)FusedBatchNorm
(CVE-2021-29583)SpaceToDepth
(CVE-2021-29587)GatherNd
(CVE-2021-29589)TransposeConv
(CVE-2021-29588)Minimum
or Maximum
(CVE-2021-29590)Reshape
operator (CVE-2021-29592)DepthToSpace
(CVE-2021-29595)EmbeddingLookup
(CVE-2021-29596)BatchToSpaceNd
(CVE-2021-29593)SpaceToBatchNd
(CVE-2021-29597)SVDF
(CVE-2021-29598)Split
(CVE-2021-29599)OneHot
(CVE-2021-29600)DepthwiseConv
(CVE-2021-29602)RaggedTensorToTensor
(CVE-2021-29608)SparseAdd
(CVE-2021-29609)SparseSparseMinimum
(CVE-2021-29607)SparseReshape
(CVE-2021-29611)QuantizeAndDequantizeV2
(CVE-2021-29610)BandedTriangularSolve
(CVE-2021-29612)tf.raw_ops.CTCLoss
(CVE-2021-29613)tf.io.decode_raw
(CVE-2021-29614)ParseAttrValue
with nested tensors (CVE-2021-29615)TrySimplify
(CVE-2021-29616)tf.transpose
with complex inputs (CVE-2021-29618)tf.strings.substr
due to CHECK
-fail (CVE-2021-29617)tf.raw_ops.SparseCountSparseOutput
(CVE-2021-29619)tf.raw_ops.ImmutableConst
(CVE-2021-29539)curl
to 7.76.0
to handle CVE-2020-8169, CVE-2020-8177, CVE-2020-8231, CVE-2020-8284, CVE-2020-8285 and CVE-2020-8286.Other
show_debug_info
to mlir.convert_graph_def
and mlir.convert_function
.--config=mkl_aarch64
build.#tensorflow #deep-learning #data-science #python