19.7. d2l API Document

Colab [mxnet]

Open the notebook in Colab

Colab [pytorch]

Open the notebook in Colab

Colab [tensorflow]

Open the notebook in Colab

SageMaker Studio Lab

Open the notebook in SageMaker Studio Lab

The implementations of the following members of the d2l package and sections where they are defined and explained can be found in the source file.

mxnetpytorchtensorflow

class d2l.mxnet.Accumulator(n)

Bases: object

For accumulating sums over n variables.

add(*args)

reset()

class d2l.mxnet.AddNorm(dropout, **kwargs)

Bases: Block

Residual connection followed by layer normalization.

Defined in Section 10.7

forward(X, Y)

Overrides to implement forward computation using NDArray. Only accepts positional arguments.

19.7. Parameters

*argslist of NDArray

Input tensors.

class d2l.mxnet.AdditiveAttention(num_hiddens, dropout, **kwargs)

Bases: Block

Additive attention.

Defined in Section 10.3

forward(queries, keys, values, valid_lens)

Overrides to implement forward computation using NDArray. Only accepts positional arguments.

19.7. Parameters

*argslist of NDArray

Input tensors.

class d2l.mxnet.Animator(xlabel=None, ylabel=None, legend=None, xlim=None, ylim=None, xscale='linear', yscale='linear', fmts=('-', 'm--', 'g-.', 'r:'), nrows=1, ncols=1, figsize=(3.5, 2.5))

Bases: object

For plotting data in animation.

add(x, y)

class d2l.mxnet.AttentionDecoder(**kwargs)

Bases: Decoder

The base attention-based decoder interface.

Defined in Section 10.4

property attention_weights

class d2l.mxnet.BERTEncoder(vocab_size, num_hiddens, ffn_num_hiddens, num_heads, num_layers, dropout, max_len=1000, **kwargs)

Bases: Block

BERT encoder.

Defined in Section 14.8.4

forward(tokens, segments, valid_lens)

Overrides to implement forward computation using NDArray. Only accepts positional arguments.

19.7. Parameters

*argslist of NDArray

Input tensors.

class d2l.mxnet.BERTModel(vocab_size, num_hiddens, ffn_num_hiddens, num_heads, num_layers, dropout, max_len=1000)

Bases: Block

The BERT model.

Defined in Section 14.8.5.2

forward(tokens, segments, valid_lens=None, pred_positions=None)

Overrides to implement forward computation using NDArray. Only accepts positional arguments.

19.7. Parameters

*argslist of NDArray

Input tensors.

class d2l.mxnet.BPRLoss(weight=None, batch_axis=0, **kwargs)

Bases: Loss

forward(positive, negative)

Defines the forward computation. Arguments can be either NDArray or Symbol.

class d2l.mxnet.BananasDataset(is_train)

Bases: Dataset

A customized dataset to load the banana detection dataset.

Defined in Section 13.6

class d2l.mxnet.Benchmark(description='Done')

Bases: object

For measuring running time.

class d2l.mxnet.CTRDataset(data_path, feat_mapper=None, defaults=None, min_threshold=4, num_feat=34)

Bases: Dataset

class d2l.mxnet.Decoder(**kwargs)

Bases: Block

The base decoder interface for the encoder-decoder architecture.

Defined in Section 9.6

forward(X, state)

Overrides to implement forward computation using NDArray. Only accepts positional arguments.

19.7. Parameters

*argslist of NDArray

Input tensors.

init_state(enc_outputs, *args)

class d2l.mxnet.DotProductAttention(dropout, **kwargs)

Bases: Block

Scaled dot product attention.

Defined in Section 10.3.2

forward(queries, keys, values, valid_lens=None)

Overrides to implement forward computation using NDArray. Only accepts positional arguments.

19.7. Parameters

*argslist of NDArray

Input tensors.

class d2l.mxnet.Encoder(**kwargs)

Bases: Block

The base encoder interface for the encoder-decoder architecture.

forward(X, *args)

Overrides to implement forward computation using NDArray. Only accepts positional arguments.

19.7. Parameters

*argslist of NDArray

Input tensors.

class d2l.mxnet.EncoderBlock(num_hiddens, ffn_num_hiddens, num_heads, dropout, use_bias=False, **kwargs)

Bases: Block

Transformer encoder block.

Defined in Section 10.7

forward(X, valid_lens)

Overrides to implement forward computation using NDArray. Only accepts positional arguments.

19.7. Parameters

*argslist of NDArray

Input tensors.

class d2l.mxnet.EncoderDecoder(encoder, decoder, **kwargs)

Bases: Block

The base class for the encoder-decoder architecture.

Defined in Section 9.6

forward(enc_X, dec_X, *args)

Overrides to implement forward computation using NDArray. Only accepts positional arguments.

19.7. Parameters

*argslist of NDArray

Input tensors.

class d2l.mxnet.HingeLossbRec(weight=None, batch_axis=0, **kwargs)

Bases: Loss

forward(positive, negative, margin=1)

Defines the forward computation. Arguments can be either NDArray or Symbol.

class d2l.mxnet.MaskLM(vocab_size, num_hiddens, **kwargs)

Bases: Block

The masked language model task of BERT.

Defined in Section 14.8.4

forward(X, pred_positions)

Overrides to implement forward computation using NDArray. Only accepts positional arguments.

19.7. Parameters

*argslist of NDArray

Input tensors.

class d2l.mxnet.MaskedSoftmaxCELoss(axis=-1, sparse_label=True, from_logits=False, weight=None, batch_axis=0, **kwargs)

Bases: SoftmaxCrossEntropyLoss

The softmax cross-entropy loss with masks.

Defined in Section 9.7.2

forward(pred, label, valid_len)

Defines the forward computation. Arguments can be either NDArray or Symbol.

class d2l.mxnet.MultiHeadAttention(num_hiddens, num_heads, dropout, use_bias=False, **kwargs)

Bases: Block

Multi-head attention.

Defined in Section 10.5

forward(queries, keys, values, valid_lens)

Overrides to implement forward computation using NDArray. Only accepts positional arguments.

19.7. Parameters

*argslist of NDArray

Input tensors.

class d2l.mxnet.NextSentencePred(**kwargs)

Bases: Block

The next sentence prediction task of BERT.

Defined in Section 14.8.5.1

forward(X)

Overrides to implement forward computation using NDArray. Only accepts positional arguments.

19.7. Parameters

*argslist of NDArray

Input tensors.

class d2l.mxnet.PositionWiseFFN(ffn_num_hiddens, ffn_num_outputs, **kwargs)

Bases: Block

Positionwise feed-forward network.

Defined in Section 10.7

forward(X)

Overrides to implement forward computation using NDArray. Only accepts positional arguments.

19.7. Parameters

*argslist of NDArray

Input tensors.

class d2l.mxnet.PositionalEncoding(num_hiddens, dropout, max_len=1000)

Bases: Block

Positional encoding.

Defined in Section 10.6

forward(X)

Overrides to implement forward computation using NDArray. Only accepts positional arguments.

19.7. Parameters

*argslist of NDArray

Input tensors.

class d2l.mxnet.RNNModel(rnn_layer, vocab_size, **kwargs)

Bases: Block

The RNN model.

Defined in Section 8.6

begin_state(*args, **kwargs)

forward(inputs, state)

Overrides to implement forward computation using NDArray. Only accepts positional arguments.

19.7. Parameters

*argslist of NDArray

Input tensors.

class d2l.mxnet.RNNModelScratch(vocab_size, num_hiddens, device, get_params, init_state, forward_fn)

Bases: object

An RNN Model implemented from scratch.

begin_state(batch_size, ctx)

class d2l.mxnet.RandomGenerator(sampling_weights)

Bases: object

Randomly draw among {1, …, n} according to n sampling weights.

draw()

class d2l.mxnet.Residual(num_channels, use_1x1conv=False, strides=1, **kwargs)

Bases: Block

The Residual block of ResNet.

forward(X)

Overrides to implement forward computation using NDArray. Only accepts positional arguments.

19.7. Parameters

*argslist of NDArray

Input tensors.

class d2l.mxnet.SNLIDataset(dataset, num_steps, vocab=None)

Bases: Dataset

A customized dataset to load the SNLI dataset.

Defined in Section 15.4

class d2l.mxnet.Seq2SeqEncoder(vocab_size, embed_size, num_hiddens, num_layers, dropout=0, **kwargs)

Bases: Encoder

The RNN encoder for sequence to sequence learning.

Defined in Section 9.7

forward(X, *args)

Overrides to implement forward computation using NDArray. Only accepts positional arguments.

19.7. Parameters

*argslist of NDArray

Input tensors.

class d2l.mxnet.SeqDataLoader(batch_size, num_steps, use_random_iter, max_tokens)

Bases: object

An iterator to load sequence data.

class d2l.mxnet.Timer

Bases: object

Record multiple running times.

avg()

Return the average time.

cumsum()

Return the accumulated time.

start()

Start the timer.

stop()

Stop the timer and record the time in a list.

sum()

Return the sum of time.

class d2l.mxnet.TokenEmbedding(embedding_name)

Bases: object

Token Embedding.

class d2l.mxnet.TransformerEncoder(vocab_size, num_hiddens, ffn_num_hiddens, num_heads, num_layers, dropout, use_bias=False, **kwargs)

Bases: Encoder

Transformer encoder.

Defined in Section 10.7

forward(X, valid_lens, *args)

Overrides to implement forward computation using NDArray. Only accepts positional arguments.

19.7. Parameters

*argslist of NDArray

Input tensors.

class d2l.mxnet.VOCSegDataset(is_train, crop_size, voc_dir)

Bases: Dataset

A customized dataset to load the VOC dataset.

Defined in Section 13.9

filter(imgs)

Returns a new dataset with samples filtered by the filter function fn.

Note that if the Dataset is the result of a lazily transformed one with transform(lazy=False), the filter is eagerly applied to the transformed samples without materializing the transformed result. That is, the transformation will be applied again whenever a sample is retrieved after filter().

19.7. Parameters

fncallable

A filter function that takes a sample as input and returns a boolean. Samples that return False are discarded.

19.7. Returns

Dataset

The filtered dataset.

normalize_image(img)

class d2l.mxnet.Vocab(tokens=None, min_freq=0, reserved_tokens=None)

Bases: object

Vocabulary for text.

to_tokens(indices)

property token_freqs

property unk

d2l.mxnet.accuracy(y_hat, y)

Compute the number of correct predictions.

Defined in Section 3.6

d2l.mxnet.annotate(text, xy, xytext)

d2l.mxnet.argmax(x, *args, **kwargs)

d2l.mxnet.assign_anchor_to_bbox(ground_truth, anchors, device, iou_threshold=0.5)

Assign closest ground-truth bounding boxes to anchor boxes.

Defined in Section 13.4

d2l.mxnet.astype(x, *args, **kwargs)

d2l.mxnet.batchify(data)

Return a minibatch of examples for skip-gram with negative sampling.

Defined in Section 14.3

d2l.mxnet.bbox_to_rect(bbox, color)

Convert bounding box to matplotlib format.

Defined in Section 13.3

d2l.mxnet.bleu(pred_seq, label_seq, k)

Compute the BLEU.

Defined in Section 9.7.4

d2l.mxnet.box_center_to_corner(boxes)

Convert from (center, width, height) to (upper-left, lower-right).

Defined in Section 13.3

d2l.mxnet.box_corner_to_center(boxes)

Convert from (upper-left, lower-right) to (center, width, height).

Defined in Section 13.3

d2l.mxnet.box_iou(boxes1, boxes2)

Compute pairwise IoU across two lists of anchor or bounding boxes.

Defined in Section 13.4

d2l.mxnet.build_array_nmt(lines, vocab, num_steps)

Transform text sequences of machine translation into minibatches.

Defined in Section 9.5.4

d2l.mxnet.copyfile(filename, target_dir)

Copy a file into a target directory.

Defined in Section 13.13

d2l.mxnet.corr2d(X, K)

Compute 2D cross-correlation.

Defined in Section 6.2

d2l.mxnet.count_corpus(tokens)

Count token frequencies.

Defined in Section 8.2

d2l.mxnet.download(name, cache_dir='../data')

Download a file inserted into DATA_HUB, return the local filename.

Defined in Section 4.10

d2l.mxnet.download_all()

Download all files in the DATA_HUB.

Defined in Section 4.10

d2l.mxnet.download_extract(name, folder=None)

Download and extract a zip/tar file.

Defined in Section 4.10

d2l.mxnet.evaluate_accuracy(net, data_iter)

Compute the accuracy for a model on a dataset.

Defined in Section 3.6

d2l.mxnet.evaluate_accuracy_gpu(net, data_iter, device=None)

Compute the accuracy for a model on a dataset using a GPU.

Defined in Section 6.6

d2l.mxnet.evaluate_accuracy_gpus(net, data_iter, split_f=<function split_batch>)

Compute the accuracy for a model on a dataset using multiple GPUs.

Defined in Section 12.6

d2l.mxnet.evaluate_loss(net, data_iter, loss)

Evaluate the loss of a model on the given dataset.

Defined in Section 4.4

d2l.mxnet.evaluate_ranking(net, test_input, seq, candidates, num_users, num_items, devices)

d2l.mxnet.get_centers_and_contexts(corpus, max_window_size)

Return center words and context words in skip-gram.

Defined in Section 14.3

d2l.mxnet.get_data_ch11(batch_size=10, n=1500)

Defined in Section 11.5.2

d2l.mxnet.get_dataloader_workers()

Use 4 processes to read the data except for Windows.

Defined in Section 3.5

d2l.mxnet.get_fashion_mnist_labels(labels)

Return text labels for the Fashion-MNIST dataset.

Defined in Section 3.5

d2l.mxnet.get_negatives(all_contexts, vocab, counter, K)

Return noise words in negative sampling.

Defined in Section 14.3

d2l.mxnet.get_tokens_and_segments(tokens_a, tokens_b=None)

Get tokens of the BERT input sequence and their segment IDs.

Defined in Section 14.8

d2l.mxnet.grad_clipping(net, theta)

Clip the gradient.

Defined in Section 8.5

d2l.mxnet.hit_and_auc(rankedlist, test_matrix, k)

d2l.mxnet.linreg(X, w, b)

The linear regression model.

Defined in Section 3.2

d2l.mxnet.load_array(data_arrays, batch_size, is_train=True)

Construct a Gluon data iterator.

Defined in Section 3.3

d2l.mxnet.load_corpus_time_machine(max_tokens=-1)

Return token indices and the vocabulary of the time machine dataset.

Defined in Section 8.2

d2l.mxnet.load_data_bananas(batch_size)

Load the banana detection dataset.

Defined in Section 13.6

d2l.mxnet.load_data_fashion_mnist(batch_size, resize=None)

Download the Fashion-MNIST dataset and then load it into memory.

Defined in Section 3.5

d2l.mxnet.load_data_imdb(batch_size, num_steps=500)

Return data iterators and the vocabulary of the IMDb review dataset.

Defined in Section 15.1

d2l.mxnet.load_data_ml100k(data, num_users, num_items, feedback='explicit')

d2l.mxnet.load_data_nmt(batch_size, num_steps, num_examples=600)

Return the iterator and the vocabularies of the translation dataset.

Defined in Section 9.5.4

d2l.mxnet.load_data_ptb(batch_size, max_window_size, num_noise_words)

Download the PTB dataset and then load it into memory.

Defined in Section 14.3.5

d2l.mxnet.load_data_snli(batch_size, num_steps=50)

Download the SNLI dataset and return data iterators and vocabulary.

Defined in Section 15.4

d2l.mxnet.load_data_time_machine(batch_size, num_steps, use_random_iter=False, max_tokens=10000)

Return the iterator and the vocabulary of the time machine dataset.

Defined in Section 8.3

d2l.mxnet.load_data_voc(batch_size, crop_size)

Load the VOC semantic segmentation dataset.

Defined in Section 13.9

d2l.mxnet.load_data_wiki(batch_size, max_len)

Load the WikiText-2 dataset.

Defined in Section 14.9.1.2

d2l.mxnet.masked_softmax(X, valid_lens)

Perform softmax operation by masking elements on the last axis.

Defined in Section 10.3

d2l.mxnet.multibox_detection(cls_probs, offset_preds, anchors, nms_threshold=0.5, pos_threshold=0.009999999)

Predict bounding boxes using non-maximum suppression.

Defined in Section 13.4.4

d2l.mxnet.multibox_prior(data, sizes, ratios)

Generate anchor boxes with different shapes centered on each pixel.

Defined in Section 13.4

d2l.mxnet.multibox_target(anchors, labels)

Label anchor boxes using ground-truth bounding boxes.

Defined in Section 13.4.3

d2l.mxnet.nms(boxes, scores, iou_threshold)

Sort confidence scores of predicted bounding boxes.

Defined in Section 13.4.4

d2l.mxnet.numpy(x, *args, **kwargs)

d2l.mxnet.offset_boxes(anchors, assigned_bb, eps=1e-06)

Transform for anchor box offsets.

Defined in Section 13.4.3

d2l.mxnet.offset_inverse(anchors, offset_preds)

Predict bounding boxes based on anchor boxes with predicted offsets.

Defined in Section 13.4.3

d2l.mxnet.plot(X, Y=None, xlabel=None, ylabel=None, legend=None, xlim=None, ylim=None, xscale='linear', yscale='linear', fmts=('-', 'm--', 'g-.', 'r:'), figsize=(3.5, 2.5), axes=None)

Plot data points.

Defined in Section 2.4

d2l.mxnet.predict_ch3(net, test_iter, n=6)

Predict labels (defined in Chapter 3).

Defined in Section 3.6

d2l.mxnet.predict_ch8(prefix, num_preds, net, vocab, device)

Generate new characters following the prefix.

Defined in Section 8.5

d2l.mxnet.predict_sentiment(net, vocab, sequence)

Predict the sentiment of a text sequence.

Defined in Section 15.2

d2l.mxnet.predict_seq2seq(net, src_sentence, src_vocab, tgt_vocab, num_steps, device, save_attention_weights=False)

Predict for sequence to sequence.

Defined in Section 9.7.4

d2l.mxnet.predict_snli(net, vocab, premise, hypothesis)

Predict the logical relationship between the premise and hypothesis.

Defined in Section 15.5

d2l.mxnet.preprocess_nmt(text)

Preprocess the English-French dataset.

Defined in Section 9.5

d2l.mxnet.read_csv_labels(fname)

Read fname to return a filename to label dictionary.

Defined in Section 13.13

d2l.mxnet.read_data_bananas(is_train=True)

Read the banana detection dataset images and labels.

Defined in Section 13.6

d2l.mxnet.read_data_ml100k()

d2l.mxnet.read_data_nmt()

Load the English-French dataset.

Defined in Section 9.5

d2l.mxnet.read_imdb(data_dir, is_train)

Read the IMDb review dataset text sequences and labels.

Defined in Section 15.1

d2l.mxnet.read_ptb()

Load the PTB dataset into a list of text lines.

Defined in Section 14.3

d2l.mxnet.read_snli(data_dir, is_train)

Read the SNLI dataset into premises, hypotheses, and labels.

Defined in Section 15.4

d2l.mxnet.read_time_machine()

Load the time machine dataset into a list of text lines.

Defined in Section 8.2

d2l.mxnet.read_voc_images(voc_dir, is_train=True)

Read all VOC feature and label images.

Defined in Section 13.9

d2l.mxnet.reduce_sum(x, *args, **kwargs)

d2l.mxnet.reorg_test(data_dir)

Organize the testing set for data loading during prediction.

Defined in Section 13.13

d2l.mxnet.reorg_train_valid(data_dir, labels, valid_ratio)

Split the validation set out of the original training set.

Defined in Section 13.13

d2l.mxnet.reshape(x, *args, **kwargs)

d2l.mxnet.resnet18(num_classes)

A slightly modified ResNet-18 model.

Defined in Section 12.6

d2l.mxnet.seq_data_iter_random(corpus, batch_size, num_steps)

Generate a minibatch of subsequences using random sampling.

Defined in Section 8.3

d2l.mxnet.seq_data_iter_sequential(corpus, batch_size, num_steps)

Generate a minibatch of subsequences using sequential partitioning.

Defined in Section 8.3

d2l.mxnet.set_axes(axes, xlabel, ylabel, xlim, ylim, xscale, yscale, legend)

Set the axes for matplotlib.

Defined in Section 2.4

d2l.mxnet.set_figsize(figsize=(3.5, 2.5))

Set the figure size for matplotlib.

Defined in Section 2.4

d2l.mxnet.sgd(params, lr, batch_size)

Minibatch stochastic gradient descent.

Defined in Section 3.2

d2l.mxnet.show_bboxes(axes, bboxes, labels=None, colors=None)

Show bounding boxes.

Defined in Section 13.4

d2l.mxnet.show_heatmaps(matrices, xlabel, ylabel, titles=None, figsize=(2.5, 2.5), cmap='Reds')

Show heatmaps of matrices.

Defined in Section 10.1

d2l.mxnet.show_images(imgs, num_rows, num_cols, titles=None, scale=1.5)

Plot a list of images.

Defined in Section 3.5

d2l.mxnet.show_list_len_pair_hist(legend, xlabel, ylabel, xlist, ylist)

Plot the histogram for list length pairs.

Defined in Section 9.5

d2l.mxnet.show_trace_2d(f, results)

Show the trace of 2D variables during optimization.

Defined in Section 11.3.1.1

d2l.mxnet.size(a)

d2l.mxnet.split_and_load_ml100k(split_mode='seq-aware', feedback='explicit', test_ratio=0.1, batch_size=256)

d2l.mxnet.split_batch(X, y, devices)

Split X and y into multiple devices.

Defined in Section 12.5

d2l.mxnet.split_batch_multi_inputs(X, y, devices)

Split multi-input X and y into multiple devices.

Defined in Section 15.5

d2l.mxnet.split_data_ml100k(data, num_users, num_items, split_mode='random', test_ratio=0.1)

Split the dataset in random mode or seq-aware mode.

d2l.mxnet.squared_loss(y_hat, y)

Squared loss.

Defined in Section 3.2

d2l.mxnet.subsample(sentences, vocab)

Subsample high-frequency words.

Defined in Section 14.3

d2l.mxnet.synthetic_data(w, b, num_examples)

Generate y = Xw + b + noise.

Defined in Section 3.2

d2l.mxnet.to(x, *args, **kwargs)

d2l.mxnet.tokenize(lines, token='word')

Split text lines into word or character tokens.

Defined in Section 8.2

d2l.mxnet.tokenize_nmt(text, num_examples=None)

Tokenize the English-French dataset.

Defined in Section 9.5

d2l.mxnet.train_2d(trainer, steps=20, f_grad=None)

Optimize a 2D objective function with a customized trainer.

Defined in Section 11.3.1.1

d2l.mxnet.train_batch_ch13(net, features, labels, loss, trainer, devices, split_f=<function split_batch>)

Train for a minibatch with mutiple GPUs (defined in Chapter 13).

Defined in Section 13.1

d2l.mxnet.train_ch11(trainer_fn, states, hyperparams, data_iter, feature_dim, num_epochs=2)

Defined in Section 11.5.2

d2l.mxnet.train_ch13(net, train_iter, test_iter, loss, trainer, num_epochs, devices=[gpu(0), gpu(1), gpu(2), gpu(3)], split_f=<function split_batch>)

Train a model with mutiple GPUs (defined in Chapter 13).

Defined in Section 13.1

d2l.mxnet.train_ch3(net, train_iter, test_iter, loss, num_epochs, updater)

Train a model (defined in Chapter 3).

Defined in Section 3.6

d2l.mxnet.train_ch6(net, train_iter, test_iter, num_epochs, lr, device)

Train a model with a GPU (defined in Chapter 6).

Defined in Section 6.6

d2l.mxnet.train_ch8(net, train_iter, vocab, lr, num_epochs, device, use_random_iter=False)

Train a model (defined in Chapter 8).

Defined in Section 8.5

d2l.mxnet.train_concise_ch11(tr_name, hyperparams, data_iter, num_epochs=2)

Defined in Section 11.5.2

d2l.mxnet.train_epoch_ch3(net, train_iter, loss, updater)

Train a model within one epoch (defined in Chapter 3).

Defined in Section 3.6

d2l.mxnet.train_epoch_ch8(net, train_iter, loss, updater, device, use_random_iter)

Train a model within one epoch (defined in Chapter 8).

Defined in Section 8.5

d2l.mxnet.train_ranking(net, train_iter, test_iter, loss, trainer, test_seq_iter, num_users, num_items, num_epochs, devices, evaluator, candidates, eval_step=1)

d2l.mxnet.train_recsys_rating(net, train_iter, test_iter, loss, trainer, num_epochs, devices=[gpu(0), gpu(1), gpu(2), gpu(3)], evaluator=None, **kwargs)

d2l.mxnet.train_seq2seq(net, data_iter, lr, num_epochs, tgt_vocab, device)

Train a model for sequence to sequence.

Defined in Section 9.7.2

d2l.mxnet.transpose(a)

d2l.mxnet.transpose_output(X, num_heads)

Reverse the operation of transpose_qkv.

Defined in Section 10.5

d2l.mxnet.transpose_qkv(X, num_heads)

Transposition for parallel computation of multiple attention heads.

Defined in Section 10.5

d2l.mxnet.truncate_pad(line, num_steps, padding_token)

Truncate or pad sequences.

Defined in Section 9.5

d2l.mxnet.try_all_gpus()

Return all available GPUs, or [cpu()] if no GPU exists.

Defined in Section 5.6

d2l.mxnet.try_gpu(i=0)

Return gpu(i) if exists, otherwise return cpu().

Defined in Section 5.6

d2l.mxnet.update_D(X, Z, net_D, net_G, loss, trainer_D)

Update discriminator.

Defined in Section 17.1

d2l.mxnet.update_G(Z, net_D, net_G, loss, trainer_G)

Update generator.

Defined in Section 17.1

d2l.mxnet.use_svg_display()

Use the svg format to display a plot in Jupyter.

Defined in Section 2.4

d2l.mxnet.voc_colormap2label()

Build the mapping from RGB to class indices for VOC labels.

Defined in Section 13.9

d2l.mxnet.voc_label_indices(colormap, colormap2label)

Map any RGB values in VOC labels to their class indices.

Defined in Section 13.9

d2l.mxnet.voc_rand_crop(feature, label, height, width)

Randomly crop both feature and label images.

Defined in Section 13.9

class d2l.torch.Accumulator(n)

Bases: object

For accumulating sums over n variables.

add(*args)

reset()

class d2l.torch.AddNorm(normalized_shape, dropout, **kwargs)

Bases: Module

Residual connection followed by layer normalization.

Defined in Section 10.7

forward(X, Y)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class d2l.torch.AdditiveAttention(key_size, query_size, num_hiddens, dropout, **kwargs)

Bases: Module

Additive attention.

Defined in Section 10.3

forward(queries, keys, values, valid_lens)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class d2l.torch.Animator(xlabel=None, ylabel=None, legend=None, xlim=None, ylim=None, xscale='linear', yscale='linear', fmts=('-', 'm--', 'g-.', 'r:'), nrows=1, ncols=1, figsize=(3.5, 2.5))

Bases: object

For plotting data in animation.

add(x, y)

class d2l.torch.AttentionDecoder(**kwargs)

Bases: Decoder

The base attention-based decoder interface.

Defined in Section 10.4

property attention_weights

training: bool

class d2l.torch.BERTEncoder(vocab_size, num_hiddens, norm_shape, ffn_num_input, ffn_num_hiddens, num_heads, num_layers, dropout, max_len=1000, key_size=768, query_size=768, value_size=768, **kwargs)

Bases: Module

BERT encoder.

Defined in Section 14.8.4

forward(tokens, segments, valid_lens)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class d2l.torch.BERTModel(vocab_size, num_hiddens, norm_shape, ffn_num_input, ffn_num_hiddens, num_heads, num_layers, dropout, max_len=1000, key_size=768, query_size=768, value_size=768, hid_in_features=768, mlm_in_features=768, nsp_in_features=768)

Bases: Module

The BERT model.

Defined in Section 14.8.5.2

forward(tokens, segments, valid_lens=None, pred_positions=None)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class d2l.torch.BananasDataset(is_train)

Bases: Dataset

A customized dataset to load the banana detection dataset.

Defined in Section 13.6

class d2l.torch.Benchmark(description='Done')

Bases: object

For measuring running time.

class d2l.torch.Decoder(**kwargs)

Bases: Module

The base decoder interface for the encoder-decoder architecture.

Defined in Section 9.6

forward(X, state)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

init_state(enc_outputs, *args)

training: bool

class d2l.torch.DotProductAttention(dropout, **kwargs)

Bases: Module

Scaled dot product attention.

Defined in Section 10.3.2

forward(queries, keys, values, valid_lens=None)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class d2l.torch.Encoder(**kwargs)

Bases: Module

The base encoder interface for the encoder-decoder architecture.

forward(X, *args)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class d2l.torch.EncoderBlock(key_size, query_size, value_size, num_hiddens, norm_shape, ffn_num_input, ffn_num_hiddens, num_heads, dropout, use_bias=False, **kwargs)

Bases: Module

Transformer encoder block.

Defined in Section 10.7

forward(X, valid_lens)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class d2l.torch.EncoderDecoder(encoder, decoder, **kwargs)

Bases: Module

The base class for the encoder-decoder architecture.

Defined in Section 9.6

forward(enc_X, dec_X, *args)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class d2l.torch.MaskLM(vocab_size, num_hiddens, num_inputs=768, **kwargs)

Bases: Module

The masked language model task of BERT.

Defined in Section 14.8.4

forward(X, pred_positions)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class d2l.torch.MaskedSoftmaxCELoss(weight: Optional[Tensor] = None, size_average=None, ignore_index: int = -100, reduce=None, reduction: str = 'mean', label_smoothing: float = 0.0)

Bases: CrossEntropyLoss

The softmax cross-entropy loss with masks.

Defined in Section 9.7.2

forward(pred, label, valid_len)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

ignore_index: int

label_smoothing: float

class d2l.torch.MultiHeadAttention(key_size, query_size, value_size, num_hiddens, num_heads, dropout, bias=False, **kwargs)

Bases: Module

Multi-head attention.

Defined in Section 10.5

forward(queries, keys, values, valid_lens)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class d2l.torch.NextSentencePred(num_inputs, **kwargs)

Bases: Module

The next sentence prediction task of BERT.

Defined in Section 14.8.5.1

forward(X)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class d2l.torch.PositionWiseFFN(ffn_num_input, ffn_num_hiddens, ffn_num_outputs, **kwargs)

Bases: Module

Positionwise feed-forward network.

Defined in Section 10.7

forward(X)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class d2l.torch.PositionalEncoding(num_hiddens, dropout, max_len=1000)

Bases: Module

Positional encoding.

Defined in Section 10.6

forward(X)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class d2l.torch.RNNModel(rnn_layer, vocab_size, **kwargs)

Bases: Module

The RNN model.

Defined in Section 8.6

begin_state(device, batch_size=1)

forward(inputs, state)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class d2l.torch.RNNModelScratch(vocab_size, num_hiddens, device, get_params, init_state, forward_fn)

Bases: object

A RNN Model implemented from scratch.

begin_state(batch_size, device)

class d2l.torch.RandomGenerator(sampling_weights)

Bases: object

Randomly draw among {1, …, n} according to n sampling weights.

draw()

class d2l.torch.Residual(input_channels, num_channels, use_1x1conv=False, strides=1)

Bases: Module

The Residual block of ResNet.

forward(X)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class d2l.torch.SNLIDataset(dataset, num_steps, vocab=None)

Bases: Dataset

A customized dataset to load the SNLI dataset.

Defined in Section 15.4

class d2l.torch.Seq2SeqEncoder(vocab_size, embed_size, num_hiddens, num_layers, dropout=0, **kwargs)

Bases: Encoder

The RNN encoder for sequence to sequence learning.

Defined in Section 9.7

forward(X, *args)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class d2l.torch.SeqDataLoader(batch_size, num_steps, use_random_iter, max_tokens)

Bases: object

An iterator to load sequence data.

class d2l.torch.Timer

Bases: object

Record multiple running times.

avg()

Return the average time.

cumsum()

Return the accumulated time.

start()

Start the timer.

stop()

Stop the timer and record the time in a list.

sum()

Return the sum of time.

class d2l.torch.TokenEmbedding(embedding_name)

Bases: object

Token Embedding.

class d2l.torch.TransformerEncoder(vocab_size, key_size, query_size, value_size, num_hiddens, norm_shape, ffn_num_input, ffn_num_hiddens, num_heads, num_layers, dropout, use_bias=False, **kwargs)

Bases: Encoder

Transformer encoder.

Defined in Section 10.7

forward(X, valid_lens, *args)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class d2l.torch.VOCSegDataset(is_train, crop_size, voc_dir)

Bases: Dataset

A customized dataset to load the VOC dataset.

Defined in Section 13.9

filter(imgs)

normalize_image(img)

class d2l.torch.Vocab(tokens=None, min_freq=0, reserved_tokens=None)

Bases: object

Vocabulary for text.

to_tokens(indices)

property token_freqs

property unk

d2l.torch.accuracy(y_hat, y)

Compute the number of correct predictions.

Defined in Section 3.6

d2l.torch.annotate(text, xy, xytext)

d2l.torch.argmax(x, *args, **kwargs)

d2l.torch.assign_anchor_to_bbox(ground_truth, anchors, device, iou_threshold=0.5)

Assign closest ground-truth bounding boxes to anchor boxes.

Defined in Section 13.4

d2l.torch.astype(x, *args, **kwargs)

d2l.torch.batchify(data)

Return a minibatch of examples for skip-gram with negative sampling.

Defined in Section 14.3

d2l.torch.bbox_to_rect(bbox, color)

Convert bounding box to matplotlib format.

Defined in Section 13.3

d2l.torch.bleu(pred_seq, label_seq, k)

Compute the BLEU.

Defined in Section 9.7.4

d2l.torch.box_center_to_corner(boxes)

Convert from (center, width, height) to (upper-left, lower-right).

Defined in Section 13.3

d2l.torch.box_corner_to_center(boxes)

Convert from (upper-left, lower-right) to (center, width, height).

Defined in Section 13.3

d2l.torch.box_iou(boxes1, boxes2)

Compute pairwise IoU across two lists of anchor or bounding boxes.

Defined in Section 13.4

d2l.torch.build_array_nmt(lines, vocab, num_steps)

Transform text sequences of machine translation into minibatches.

Defined in Section 9.5.4

d2l.torch.copyfile(filename, target_dir)

Copy a file into a target directory.

Defined in Section 13.13

d2l.torch.corr2d(X, K)

Compute 2D cross-correlation.

Defined in Section 6.2

d2l.torch.count_corpus(tokens)

Count token frequencies.

Defined in Section 8.2

d2l.torch.download(name, cache_dir='../data')

Download a file inserted into DATA_HUB, return the local filename.

Defined in Section 4.10

d2l.torch.download_all()

Download all files in the DATA_HUB.

Defined in Section 4.10

d2l.torch.download_extract(name, folder=None)

Download and extract a zip/tar file.

Defined in Section 4.10

d2l.torch.evaluate_accuracy(net, data_iter)

Compute the accuracy for a model on a dataset.

Defined in Section 3.6

d2l.torch.evaluate_accuracy_gpu(net, data_iter, device=None)

Compute the accuracy for a model on a dataset using a GPU.

Defined in Section 6.6

d2l.torch.evaluate_loss(net, data_iter, loss)

Evaluate the loss of a model on the given dataset.

Defined in Section 4.4

d2l.torch.get_centers_and_contexts(corpus, max_window_size)

Return center words and context words in skip-gram.

Defined in Section 14.3

d2l.torch.get_data_ch11(batch_size=10, n=1500)

Defined in Section 11.5.2

d2l.torch.get_dataloader_workers()

Use 4 processes to read the data.

Defined in Section 3.5

d2l.torch.get_fashion_mnist_labels(labels)

Return text labels for the Fashion-MNIST dataset.

Defined in Section 3.5

d2l.torch.get_negatives(all_contexts, vocab, counter, K)

Return noise words in negative sampling.

Defined in Section 14.3

d2l.torch.get_tokens_and_segments(tokens_a, tokens_b=None)

Get tokens of the BERT input sequence and their segment IDs.

Defined in Section 14.8

d2l.torch.grad_clipping(net, theta)

Clip the gradient.

Defined in Section 8.5

d2l.torch.linreg(X, w, b)

The linear regression model.

Defined in Section 3.2

d2l.torch.load_array(data_arrays, batch_size, is_train=True)

Construct a PyTorch data iterator.

Defined in Section 3.3

d2l.torch.load_corpus_time_machine(max_tokens=-1)

Return token indices and the vocabulary of the time machine dataset.

Defined in Section 8.2

d2l.torch.load_data_bananas(batch_size)

Load the banana detection dataset.

Defined in Section 13.6

d2l.torch.load_data_fashion_mnist(batch_size, resize=None)

Download the Fashion-MNIST dataset and then load it into memory.

Defined in Section 3.5

d2l.torch.load_data_imdb(batch_size, num_steps=500)

Return data iterators and the vocabulary of the IMDb review dataset.

Defined in Section 15.1

d2l.torch.load_data_nmt(batch_size, num_steps, num_examples=600)

Return the iterator and the vocabularies of the translation dataset.

Defined in Section 9.5.4

d2l.torch.load_data_ptb(batch_size, max_window_size, num_noise_words)

Download the PTB dataset and then load it into memory.

Defined in Section 14.3.5

d2l.torch.load_data_snli(batch_size, num_steps=50)

Download the SNLI dataset and return data iterators and vocabulary.

Defined in Section 15.4

d2l.torch.load_data_time_machine(batch_size, num_steps, use_random_iter=False, max_tokens=10000)

Return the iterator and the vocabulary of the time machine dataset.

Defined in Section 8.3

d2l.torch.load_data_voc(batch_size, crop_size)

Load the VOC semantic segmentation dataset.

Defined in Section 13.9

d2l.torch.load_data_wiki(batch_size, max_len)

Load the WikiText-2 dataset.

Defined in Section 14.9.1.2

d2l.torch.masked_softmax(X, valid_lens)

Perform softmax operation by masking elements on the last axis.

Defined in Section 10.3

d2l.torch.multibox_detection(cls_probs, offset_preds, anchors, nms_threshold=0.5, pos_threshold=0.009999999)

Predict bounding boxes using non-maximum suppression.

Defined in Section 13.4.4

d2l.torch.multibox_prior(data, sizes, ratios)

Generate anchor boxes with different shapes centered on each pixel.

Defined in Section 13.4

d2l.torch.multibox_target(anchors, labels)

Label anchor boxes using ground-truth bounding boxes.

Defined in Section 13.4.3

d2l.torch.nms(boxes, scores, iou_threshold)

Sort confidence scores of predicted bounding boxes.

Defined in Section 13.4.4

d2l.torch.numpy(x, *args, **kwargs)

d2l.torch.offset_boxes(anchors, assigned_bb, eps=1e-06)

Transform for anchor box offsets.

Defined in Section 13.4.3

d2l.torch.offset_inverse(anchors, offset_preds)

Predict bounding boxes based on anchor boxes with predicted offsets.

Defined in Section 13.4.3

d2l.torch.plot(X, Y=None, xlabel=None, ylabel=None, legend=None, xlim=None, ylim=None, xscale='linear', yscale='linear', fmts=('-', 'm--', 'g-.', 'r:'), figsize=(3.5, 2.5), axes=None)

Plot data points.

Defined in Section 2.4

d2l.torch.predict_ch3(net, test_iter, n=6)

Predict labels (defined in Chapter 3).

Defined in Section 3.6

d2l.torch.predict_ch8(prefix, num_preds, net, vocab, device)

Generate new characters following the prefix.

Defined in Section 8.5

d2l.torch.predict_sentiment(net, vocab, sequence)

Predict the sentiment of a text sequence.

Defined in Section 15.2

d2l.torch.predict_seq2seq(net, src_sentence, src_vocab, tgt_vocab, num_steps, device, save_attention_weights=False)

Predict for sequence to sequence.

Defined in Section 9.7.4

d2l.torch.predict_snli(net, vocab, premise, hypothesis)

Predict the logical relationship between the premise and hypothesis.

Defined in Section 15.5

d2l.torch.preprocess_nmt(text)

Preprocess the English-French dataset.

Defined in Section 9.5

d2l.torch.read_csv_labels(fname)

Read fname to return a filename to label dictionary.

Defined in Section 13.13

d2l.torch.read_data_bananas(is_train=True)

Read the banana detection dataset images and labels.

Defined in Section 13.6

d2l.torch.read_data_nmt()

Load the English-French dataset.

Defined in Section 9.5

d2l.torch.read_imdb(data_dir, is_train)

Read the IMDb review dataset text sequences and labels.

Defined in Section 15.1

d2l.torch.read_ptb()

Load the PTB dataset into a list of text lines.

Defined in Section 14.3

d2l.torch.read_snli(data_dir, is_train)

Read the SNLI dataset into premises, hypotheses, and labels.

Defined in Section 15.4

d2l.torch.read_time_machine()

Load the time machine dataset into a list of text lines.

Defined in Section 8.2

d2l.torch.read_voc_images(voc_dir, is_train=True)

Read all VOC feature and label images.

Defined in Section 13.9

d2l.torch.reduce_sum(x, *args, **kwargs)

d2l.torch.reorg_test(data_dir)

Organize the testing set for data loading during prediction.

Defined in Section 13.13

d2l.torch.reorg_train_valid(data_dir, labels, valid_ratio)

Split the validation set out of the original training set.

Defined in Section 13.13

d2l.torch.reshape(x, *args, **kwargs)

d2l.torch.resnet18(num_classes, in_channels=1)

A slightly modified ResNet-18 model.

Defined in Section 12.6

d2l.torch.seq_data_iter_random(corpus, batch_size, num_steps)

Generate a minibatch of subsequences using random sampling.

Defined in Section 8.3

d2l.torch.seq_data_iter_sequential(corpus, batch_size, num_steps)

Generate a minibatch of subsequences using sequential partitioning.

Defined in Section 8.3

d2l.torch.sequence_mask(X, valid_len, value=0)

Mask irrelevant entries in sequences.

Defined in Section 9.7.2

d2l.torch.set_axes(axes, xlabel, ylabel, xlim, ylim, xscale, yscale, legend)

Set the axes for matplotlib.

Defined in Section 2.4

d2l.torch.set_figsize(figsize=(3.5, 2.5))

Set the figure size for matplotlib.

Defined in Section 2.4

d2l.torch.sgd(params, lr, batch_size)

Minibatch stochastic gradient descent.

Defined in Section 3.2

d2l.torch.show_bboxes(axes, bboxes, labels=None, colors=None)

Show bounding boxes.

Defined in Section 13.4

d2l.torch.show_heatmaps(matrices, xlabel, ylabel, titles=None, figsize=(2.5, 2.5), cmap='Reds')

Show heatmaps of matrices.

Defined in Section 10.1

d2l.torch.show_images(imgs, num_rows, num_cols, titles=None, scale=1.5)

Plot a list of images.

Defined in Section 3.5

d2l.torch.show_list_len_pair_hist(legend, xlabel, ylabel, xlist, ylist)

Plot the histogram for list length pairs.

Defined in Section 9.5

d2l.torch.show_trace_2d(f, results)

Show the trace of 2D variables during optimization.

Defined in Section 11.3.1.1

d2l.torch.size(x, *args, **kwargs)

d2l.torch.split_batch(X, y, devices)

Split X and y into multiple devices.

Defined in Section 12.5

d2l.torch.squared_loss(y_hat, y)

Squared loss.

Defined in Section 3.2

d2l.torch.subsample(sentences, vocab)

Subsample high-frequency words.

Defined in Section 14.3

d2l.torch.synthetic_data(w, b, num_examples)

Generate y = Xw + b + noise.

Defined in Section 3.2

d2l.torch.to(x, *args, **kwargs)

d2l.torch.tokenize(lines, token='word')

Split text lines into word or character tokens.

Defined in Section 8.2

d2l.torch.tokenize_nmt(text, num_examples=None)

Tokenize the English-French dataset.

Defined in Section 9.5

d2l.torch.train_2d(trainer, steps=20, f_grad=None)

Optimize a 2D objective function with a customized trainer.

Defined in Section 11.3.1.1

d2l.torch.train_batch_ch13(net, X, y, loss, trainer, devices)

Train for a minibatch with mutiple GPUs (defined in Chapter 13).

Defined in Section 13.1

d2l.torch.train_ch11(trainer_fn, states, hyperparams, data_iter, feature_dim, num_epochs=2)

Defined in Section 11.5.2

d2l.torch.train_ch13(net, train_iter, test_iter, loss, trainer, num_epochs, devices=[device(type='cuda', index=0), device(type='cuda', index=1), device(type='cuda', index=2), device(type='cuda', index=3)])

Train a model with mutiple GPUs (defined in Chapter 13).

Defined in Section 13.1

d2l.torch.train_ch3(net, train_iter, test_iter, loss, num_epochs, updater)

Train a model (defined in Chapter 3).

Defined in Section 3.6

d2l.torch.train_ch6(net, train_iter, test_iter, num_epochs, lr, device)

Train a model with a GPU (defined in Chapter 6).

Defined in Section 6.6

d2l.torch.train_ch8(net, train_iter, vocab, lr, num_epochs, device, use_random_iter=False)

Train a model (defined in Chapter 8).

Defined in Section 8.5

d2l.torch.train_concise_ch11(trainer_fn, hyperparams, data_iter, num_epochs=4)

Defined in Section 11.5.2

d2l.torch.train_epoch_ch3(net, train_iter, loss, updater)

The training loop defined in Chapter 3.

Defined in Section 3.6

d2l.torch.train_epoch_ch8(net, train_iter, loss, updater, device, use_random_iter)

Train a net within one epoch (defined in Chapter 8).

Defined in Section 8.5

d2l.torch.train_seq2seq(net, data_iter, lr, num_epochs, tgt_vocab, device)

Train a model for sequence to sequence.

Defined in Section 9.7.2

d2l.torch.transpose(x, *args, **kwargs)

d2l.torch.transpose_output(X, num_heads)

Reverse the operation of transpose_qkv.

Defined in Section 10.5

d2l.torch.transpose_qkv(X, num_heads)

Transposition for parallel computation of multiple attention heads.

Defined in Section 10.5

d2l.torch.truncate_pad(line, num_steps, padding_token)

Truncate or pad sequences.

Defined in Section 9.5

d2l.torch.try_all_gpus()

Return all available GPUs, or [cpu(),] if no GPU exists.

Defined in Section 5.6

d2l.torch.try_gpu(i=0)

Return gpu(i) if exists, otherwise return cpu().

Defined in Section 5.6

d2l.torch.update_D(X, Z, net_D, net_G, loss, trainer_D)

Update discriminator.

Defined in Section 17.1

d2l.torch.update_G(Z, net_D, net_G, loss, trainer_G)

Update generator.

Defined in Section 17.1

d2l.torch.use_svg_display()

Use the svg format to display a plot in Jupyter.

Defined in Section 2.4

d2l.torch.voc_colormap2label()

Build the mapping from RGB to class indices for VOC labels.

Defined in Section 13.9

d2l.torch.voc_label_indices(colormap, colormap2label)

Map any RGB values in VOC labels to their class indices.

Defined in Section 13.9

d2l.torch.voc_rand_crop(feature, label, height, width)

Randomly crop both feature and label images.

Defined in Section 13.9

class d2l.tensorflow.Accumulator(n)

Bases: object

For accumulating sums over n variables.

add(*args)

reset()

class d2l.tensorflow.AddNorm(*args, **kwargs)

Bases: Layer

Residual connection followed by layer normalization.

Defined in Section 10.7

call(X, Y, **kwargs)

This is where the layer’s logic lives.

The call() method may not create state (except in its first invocation, wrapping the creation of variables or other resources in tf.init_scope()). It is recommended to create state in __init__(), or the build() method that is called automatically before call() executes the first time.

Args:

inputs: Input tensor, or dict/list/tuple of input tensors.

The first positional inputs argument is subject to special rules: - inputs must be explicitly passed. A layer cannot have zero

arguments, and inputs cannot be provided via the default value of a keyword argument.

• NumPy array or Python scalar values in inputs get cast as tensors.

• Keras mask metadata is only collected from inputs.

• Layers are built (build(input_shape) method) using shape info from inputs only.

• input_spec compatibility is only checked against inputs.

• Mixed precision input casting is only applied to inputs. If a layer has tensor arguments in *args or **kwargs, their casting behavior in mixed precision should be handled manually.

• The SavedModel input specification is generated using inputs only.

• Integration with various ecosystem packages like TFMOT, TFLite, TF.js, etc is only supported for inputs and not for tensors in positional and keyword arguments.

*args: Additional positional arguments. May contain tensors, although

this is not recommended, for the reasons above.

**kwargs: Additional keyword arguments. May contain tensors, although

this is not recommended, for the reasons above. The following optional keyword arguments are reserved: - training: Boolean scalar tensor of Python boolean indicating

whether the call is meant for training or inference.

• mask: Boolean input mask. If the layer’s call() method takes a mask argument, its default value will be set to the mask generated for inputs by the previous layer (if input did come from a layer that generated a corresponding mask, i.e. if it came from a Keras layer with masking support).

Returns:

A tensor or list/tuple of tensors.

class d2l.tensorflow.AdditiveAttention(*args, **kwargs)

Bases: Layer

Additive attention.

Defined in Section 10.3

call(queries, keys, values, valid_lens, **kwargs)

This is where the layer’s logic lives.

The call() method may not create state (except in its first invocation, wrapping the creation of variables or other resources in tf.init_scope()). It is recommended to create state in __init__(), or the build() method that is called automatically before call() executes the first time.

Args:

inputs: Input tensor, or dict/list/tuple of input tensors.

The first positional inputs argument is subject to special rules: - inputs must be explicitly passed. A layer cannot have zero

arguments, and inputs cannot be provided via the default value of a keyword argument.

• NumPy array or Python scalar values in inputs get cast as tensors.

• Keras mask metadata is only collected from inputs.

• Layers are built (build(input_shape) method) using shape info from inputs only.

• input_spec compatibility is only checked against inputs.

• Mixed precision input casting is only applied to inputs. If a layer has tensor arguments in *args or **kwargs, their casting behavior in mixed precision should be handled manually.

• The SavedModel input specification is generated using inputs only.

• Integration with various ecosystem packages like TFMOT, TFLite, TF.js, etc is only supported for inputs and not for tensors in positional and keyword arguments.

*args: Additional positional arguments. May contain tensors, although

this is not recommended, for the reasons above.

**kwargs: Additional keyword arguments. May contain tensors, although

this is not recommended, for the reasons above. The following optional keyword arguments are reserved: - training: Boolean scalar tensor of Python boolean indicating

whether the call is meant for training or inference.

• mask: Boolean input mask. If the layer’s call() method takes a mask argument, its default value will be set to the mask generated for inputs by the previous layer (if input did come from a layer that generated a corresponding mask, i.e. if it came from a Keras layer with masking support).

Returns:

A tensor or list/tuple of tensors.

class d2l.tensorflow.Animator(xlabel=None, ylabel=None, legend=None, xlim=None, ylim=None, xscale='linear', yscale='linear', fmts=('-', 'm--', 'g-.', 'r:'), nrows=1, ncols=1, figsize=(3.5, 2.5))

Bases: object

For plotting data in animation.

add(x, y)

class d2l.tensorflow.AttentionDecoder(*args, **kwargs)

Bases: Decoder

The base attention-based decoder interface.

Defined in Section 10.4

property attention_weights

class d2l.tensorflow.Benchmark(description='Done')

Bases: object

For measuring running time.

class d2l.tensorflow.Decoder(*args, **kwargs)

Bases: Layer

The base decoder interface for the encoder-decoder architecture.

Defined in Section 9.6

call(X, state, **kwargs)

This is where the layer’s logic lives.

The call() method may not create state (except in its first invocation, wrapping the creation of variables or other resources in tf.init_scope()). It is recommended to create state in __init__(), or the build() method that is called automatically before call() executes the first time.

Args:

inputs: Input tensor, or dict/list/tuple of input tensors.

The first positional inputs argument is subject to special rules: - inputs must be explicitly passed. A layer cannot have zero

arguments, and inputs cannot be provided via the default value of a keyword argument.

• NumPy array or Python scalar values in inputs get cast as tensors.

• Keras mask metadata is only collected from inputs.

• Layers are built (build(input_shape) method) using shape info from inputs only.

• input_spec compatibility is only checked against inputs.

• Mixed precision input casting is only applied to inputs. If a layer has tensor arguments in *args or **kwargs, their casting behavior in mixed precision should be handled manually.

• The SavedModel input specification is generated using inputs only.

• Integration with various ecosystem packages like TFMOT, TFLite, TF.js, etc is only supported for inputs and not for tensors in positional and keyword arguments.

*args: Additional positional arguments. May contain tensors, although

this is not recommended, for the reasons above.

**kwargs: Additional keyword arguments. May contain tensors, although

this is not recommended, for the reasons above. The following optional keyword arguments are reserved: - training: Boolean scalar tensor of Python boolean indicating

whether the call is meant for training or inference.

• mask: Boolean input mask. If the layer’s call() method takes a mask argument, its default value will be set to the mask generated for inputs by the previous layer (if input did come from a layer that generated a corresponding mask, i.e. if it came from a Keras layer with masking support).

Returns:

A tensor or list/tuple of tensors.

init_state(enc_outputs, *args)

class d2l.tensorflow.DotProductAttention(*args, **kwargs)

Bases: Layer

Scaled dot product attention.

Defined in Section 10.3.2

call(queries, keys, values, valid_lens, **kwargs)

This is where the layer’s logic lives.

The call() method may not create state (except in its first invocation, wrapping the creation of variables or other resources in tf.init_scope()). It is recommended to create state in __init__(), or the build() method that is called automatically before call() executes the first time.

Args:

inputs: Input tensor, or dict/list/tuple of input tensors.

The first positional inputs argument is subject to special rules: - inputs must be explicitly passed. A layer cannot have zero

arguments, and inputs cannot be provided via the default value of a keyword argument.

• NumPy array or Python scalar values in inputs get cast as tensors.

• Keras mask metadata is only collected from inputs.

• Layers are built (build(input_shape) method) using shape info from inputs only.

• input_spec compatibility is only checked against inputs.

• Mixed precision input casting is only applied to inputs. If a layer has tensor arguments in *args or **kwargs, their casting behavior in mixed precision should be handled manually.

• The SavedModel input specification is generated using inputs only.

• Integration with various ecosystem packages like TFMOT, TFLite, TF.js, etc is only supported for inputs and not for tensors in positional and keyword arguments.

*args: Additional positional arguments. May contain tensors, although

this is not recommended, for the reasons above.

**kwargs: Additional keyword arguments. May contain tensors, although

this is not recommended, for the reasons above. The following optional keyword arguments are reserved: - training: Boolean scalar tensor of Python boolean indicating

whether the call is meant for training or inference.

• mask: Boolean input mask. If the layer’s call() method takes a mask argument, its default value will be set to the mask generated for inputs by the previous layer (if input did come from a layer that generated a corresponding mask, i.e. if it came from a Keras layer with masking support).

Returns:

A tensor or list/tuple of tensors.

class d2l.tensorflow.Encoder(*args, **kwargs)

Bases: Layer

The base encoder interface for the encoder-decoder architecture.

call(X, *args, **kwargs)

This is where the layer’s logic lives.

The call() method may not create state (except in its first invocation, wrapping the creation of variables or other resources in tf.init_scope()). It is recommended to create state in __init__(), or the build() method that is called automatically before call() executes the first time.

Args:

inputs: Input tensor, or dict/list/tuple of input tensors.

The first positional inputs argument is subject to special rules: - inputs must be explicitly passed. A layer cannot have zero

arguments, and inputs cannot be provided via the default value of a keyword argument.

• NumPy array or Python scalar values in inputs get cast as tensors.

• Keras mask metadata is only collected from inputs.

• Layers are built (build(input_shape) method) using shape info from inputs only.

• input_spec compatibility is only checked against inputs.

• Mixed precision input casting is only applied to inputs. If a layer has tensor arguments in *args or **kwargs, their casting behavior in mixed precision should be handled manually.

• The SavedModel input specification is generated using inputs only.

• Integration with various ecosystem packages like TFMOT, TFLite, TF.js, etc is only supported for inputs and not for tensors in positional and keyword arguments.

*args: Additional positional arguments. May contain tensors, although

this is not recommended, for the reasons above.

**kwargs: Additional keyword arguments. May contain tensors, although

this is not recommended, for the reasons above. The following optional keyword arguments are reserved: - training: Boolean scalar tensor of Python boolean indicating

whether the call is meant for training or inference.

• mask: Boolean input mask. If the layer’s call() method takes a mask argument, its default value will be set to the mask generated for inputs by the previous layer (if input did come from a layer that generated a corresponding mask, i.e. if it came from a Keras layer with masking support).

Returns:

A tensor or list/tuple of tensors.

class d2l.tensorflow.EncoderBlock(*args, **kwargs)

Bases: Layer

Transformer encoder block.

Defined in Section 10.7

call(X, valid_lens, **kwargs)

This is where the layer’s logic lives.

The call() method may not create state (except in its first invocation, wrapping the creation of variables or other resources in tf.init_scope()). It is recommended to create state in __init__(), or the build() method that is called automatically before call() executes the first time.

Args:

inputs: Input tensor, or dict/list/tuple of input tensors.

The first positional inputs argument is subject to special rules: - inputs must be explicitly passed. A layer cannot have zero

arguments, and inputs cannot be provided via the default value of a keyword argument.

• NumPy array or Python scalar values in inputs get cast as tensors.

• Keras mask metadata is only collected from inputs.

• Layers are built (build(input_shape) method) using shape info from inputs only.

• input_spec compatibility is only checked against inputs.

• Mixed precision input casting is only applied to inputs. If a layer has tensor arguments in *args or **kwargs, their casting behavior in mixed precision should be handled manually.

• The SavedModel input specification is generated using inputs only.

• Integration with various ecosystem packages like TFMOT, TFLite, TF.js, etc is only supported for inputs and not for tensors in positional and keyword arguments.

*args: Additional positional arguments. May contain tensors, although

this is not recommended, for the reasons above.

**kwargs: Additional keyword arguments. May contain tensors, although

this is not recommended, for the reasons above. The following optional keyword arguments are reserved: - training: Boolean scalar tensor of Python boolean indicating

whether the call is meant for training or inference.

• mask: Boolean input mask. If the layer’s call() method takes a mask argument, its default value will be set to the mask generated for inputs by the previous layer (if input did come from a layer that generated a corresponding mask, i.e. if it came from a Keras layer with masking support).

Returns:

A tensor or list/tuple of tensors.

class d2l.tensorflow.EncoderDecoder(*args, **kwargs)

Bases: Model

The base class for the encoder-decoder architecture.

Defined in Section 9.6

call(enc_X, dec_X, *args, **kwargs)

Calls the model on new inputs and returns the outputs as tensors.

In this case call() just reapplies all ops in the graph to the new inputs (e.g. build a new computational graph from the provided inputs).

Note: This method should not be called directly. It is only meant to be overridden when subclassing tf.keras.Model. To call a model on an input, always use the __call__() method, i.e. model(inputs), which relies on the underlying call() method.

Args:

inputs: Input tensor, or dict/list/tuple of input tensors. training: Boolean or boolean scalar tensor, indicating whether to run

the Network in training mode or inference mode.

mask: A mask or list of masks. A mask can be either a boolean tensor or

None (no mask). For more details, check the guide

[here](https://www.tensorflow.org/guide/keras/masking_and_padding).

Returns:

A tensor if there is a single output, or a list of tensors if there are more than one outputs.

class d2l.tensorflow.MaskedSoftmaxCELoss(valid_len)

Bases: Loss

The softmax cross-entropy loss with masks.

Defined in Section 9.7.2

call(label, pred)

Invokes the Loss instance.

Args:

y_true: Ground truth values. shape = [batch_size, d0, .. dN], except

sparse loss functions such as sparse categorical crossentropy where shape = [batch_size, d0, .. dN-1]

y_pred: The predicted values. shape = [batch_size, d0, .. dN]

Returns:

Loss values with the shape [batch_size, d0, .. dN-1].

class d2l.tensorflow.MultiHeadAttention(*args, **kwargs)

Bases: Layer

Multi-head attention.

Defined in Section 10.5

call(queries, keys, values, valid_lens, **kwargs)

This is where the layer’s logic lives.

The call() method may not create state (except in its first invocation, wrapping the creation of variables or other resources in tf.init_scope()). It is recommended to create state in __init__(), or the build() method that is called automatically before call() executes the first time.

Args:

inputs: Input tensor, or dict/list/tuple of input tensors.

The first positional inputs argument is subject to special rules: - inputs must be explicitly passed. A layer cannot have zero

arguments, and inputs cannot be provided via the default value of a keyword argument.

• NumPy array or Python scalar values in inputs get cast as tensors.

• Keras mask metadata is only collected from inputs.

• Layers are built (build(input_shape) method) using shape info from inputs only.

• input_spec compatibility is only checked against inputs.

• Mixed precision input casting is only applied to inputs. If a layer has tensor arguments in *args or **kwargs, their casting behavior in mixed precision should be handled manually.

• The SavedModel input specification is generated using inputs only.

• Integration with various ecosystem packages like TFMOT, TFLite, TF.js, etc is only supported for inputs and not for tensors in positional and keyword arguments.

*args: Additional positional arguments. May contain tensors, although

this is not recommended, for the reasons above.

**kwargs: Additional keyword arguments. May contain tensors, although

this is not recommended, for the reasons above. The following optional keyword arguments are reserved: - training: Boolean scalar tensor of Python boolean indicating

whether the call is meant for training or inference.

• mask: Boolean input mask. If the layer’s call() method takes a mask argument, its default value will be set to the mask generated for inputs by the previous layer (if input did come from a layer that generated a corresponding mask, i.e. if it came from a Keras layer with masking support).

Returns:

A tensor or list/tuple of tensors.

class d2l.tensorflow.PositionWiseFFN(*args, **kwargs)

Bases: Layer

Positionwise feed-forward network.

Defined in Section 10.7

call(X)

This is where the layer’s logic lives.

The call() method may not create state (except in its first invocation, wrapping the creation of variables or other resources in tf.init_scope()). It is recommended to create state in __init__(), or the build() method that is called automatically before call() executes the first time.

Args:

inputs: Input tensor, or dict/list/tuple of input tensors.

The first positional inputs argument is subject to special rules: - inputs must be explicitly passed. A layer cannot have zero

arguments, and inputs cannot be provided via the default value of a keyword argument.

• NumPy array or Python scalar values in inputs get cast as tensors.

• Keras mask metadata is only collected from inputs.

• Layers are built (build(input_shape) method) using shape info from inputs only.

• input_spec compatibility is only checked against inputs.

• Mixed precision input casting is only applied to inputs. If a layer has tensor arguments in *args or **kwargs, their casting behavior in mixed precision should be handled manually.

• The SavedModel input specification is generated using inputs only.

• Integration with various ecosystem packages like TFMOT, TFLite, TF.js, etc is only supported for inputs and not for tensors in positional and keyword arguments.

*args: Additional positional arguments. May contain tensors, although

this is not recommended, for the reasons above.

**kwargs: Additional keyword arguments. May contain tensors, although

this is not recommended, for the reasons above. The following optional keyword arguments are reserved: - training: Boolean scalar tensor of Python boolean indicating

whether the call is meant for training or inference.

• mask: Boolean input mask. If the layer’s call() method takes a mask argument, its default value will be set to the mask generated for inputs by the previous layer (if input did come from a layer that generated a corresponding mask, i.e. if it came from a Keras layer with masking support).

Returns:

A tensor or list/tuple of tensors.

class d2l.tensorflow.PositionalEncoding(*args, **kwargs)

Bases: Layer

Positional encoding.

Defined in Section 10.6

call(X, **kwargs)

This is where the layer’s logic lives.

The call() method may not create state (except in its first invocation, wrapping the creation of variables or other resources in tf.init_scope()). It is recommended to create state in __init__(), or the build() method that is called automatically before call() executes the first time.

Args:

inputs: Input tensor, or dict/list/tuple of input tensors.

The first positional inputs argument is subject to special rules: - inputs must be explicitly passed. A layer cannot have zero

arguments, and inputs cannot be provided via the default value of a keyword argument.

• NumPy array or Python scalar values in inputs get cast as tensors.

• Keras mask metadata is only collected from inputs.

• Layers are built (build(input_shape) method) using shape info from inputs only.

• input_spec compatibility is only checked against inputs.

• Mixed precision input casting is only applied to inputs. If a layer has tensor arguments in *args or **kwargs, their casting behavior in mixed precision should be handled manually.

• The SavedModel input specification is generated using inputs only.

• Integration with various ecosystem packages like TFMOT, TFLite, TF.js, etc is only supported for inputs and not for tensors in positional and keyword arguments.

*args: Additional positional arguments. May contain tensors, although

this is not recommended, for the reasons above.

**kwargs: Additional keyword arguments. May contain tensors, although

this is not recommended, for the reasons above. The following optional keyword arguments are reserved: - training: Boolean scalar tensor of Python boolean indicating

whether the call is meant for training or inference.

• mask: Boolean input mask. If the layer’s call() method takes a mask argument, its default value will be set to the mask generated for inputs by the previous layer (if input did come from a layer that generated a corresponding mask, i.e. if it came from a Keras layer with masking support).

Returns:

A tensor or list/tuple of tensors.

class d2l.tensorflow.RNNModel(*args, **kwargs)

Bases: Layer

Defined in Section 8.6

begin_state(*args, **kwargs)

call(inputs, state)

This is where the layer’s logic lives.

The call() method may not create state (except in its first invocation, wrapping the creation of variables or other resources in tf.init_scope()). It is recommended to create state in __init__(), or the build() method that is called automatically before call() executes the first time.

Args:

inputs: Input tensor, or dict/list/tuple of input tensors.

The first positional inputs argument is subject to special rules: - inputs must be explicitly passed. A layer cannot have zero

arguments, and inputs cannot be provided via the default value of a keyword argument.

• NumPy array or Python scalar values in inputs get cast as tensors.

• Keras mask metadata is only collected from inputs.

• Layers are built (build(input_shape) method) using shape info from inputs only.

• input_spec compatibility is only checked against inputs.

• Mixed precision input casting is only applied to inputs. If a layer has tensor arguments in *args or **kwargs, their casting behavior in mixed precision should be handled manually.

• The SavedModel input specification is generated using inputs only.

• Integration with various ecosystem packages like TFMOT, TFLite, TF.js, etc is only supported for inputs and not for tensors in positional and keyword arguments.

*args: Additional positional arguments. May contain tensors, although

this is not recommended, for the reasons above.

**kwargs: Additional keyword arguments. May contain tensors, although

this is not recommended, for the reasons above. The following optional keyword arguments are reserved: - training: Boolean scalar tensor of Python boolean indicating

whether the call is meant for training or inference.

• mask: Boolean input mask. If the layer’s call() method takes a mask argument, its default value will be set to the mask generated for inputs by the previous layer (if input did come from a layer that generated a corresponding mask, i.e. if it came from a Keras layer with masking support).

Returns:

A tensor or list/tuple of tensors.

class d2l.tensorflow.RNNModelScratch(vocab_size, num_hiddens, init_state, forward_fn, get_params)

Bases: object

A RNN Model implemented from scratch.

begin_state(batch_size, *args, **kwargs)

class d2l.tensorflow.Residual(*args, **kwargs)

Bases: Model

The Residual block of ResNet.

call(X)

Calls the model on new inputs and returns the outputs as tensors.

In this case call() just reapplies all ops in the graph to the new inputs (e.g. build a new computational graph from the provided inputs).

Note: This method should not be called directly. It is only meant to be overridden when subclassing tf.keras.Model. To call a model on an input, always use the __call__() method, i.e. model(inputs), which relies on the underlying call() method.

Args:

inputs: Input tensor, or dict/list/tuple of input tensors. training: Boolean or boolean scalar tensor, indicating whether to run

the Network in training mode or inference mode.

mask: A mask or list of masks. A mask can be either a boolean tensor or

None (no mask). For more details, check the guide

[here](https://www.tensorflow.org/guide/keras/masking_and_padding).

Returns:

A tensor if there is a single output, or a list of tensors if there are more than one outputs.

class d2l.tensorflow.Seq2SeqEncoder(*args, **kwargs)

Bases: Encoder

The RNN encoder for sequence to sequence learning.

Defined in Section 9.7

call(X, *args, **kwargs)

This is where the layer’s logic lives.

The call() method may not create state (except in its first invocation, wrapping the creation of variables or other resources in tf.init_scope()). It is recommended to create state in __init__(), or the build() method that is called automatically before call() executes the first time.

Args:

inputs: Input tensor, or dict/list/tuple of input tensors.

The first positional inputs argument is subject to special rules: - inputs must be explicitly passed. A layer cannot have zero

arguments, and inputs cannot be provided via the default value of a keyword argument.

• NumPy array or Python scalar values in inputs get cast as tensors.

• Keras mask metadata is only collected from inputs.

• Layers are built (build(input_shape) method) using shape info from inputs only.

• input_spec compatibility is only checked against inputs.

• Mixed precision input casting is only applied to inputs. If a layer has tensor arguments in *args or **kwargs, their casting behavior in mixed precision should be handled manually.

• The SavedModel input specification is generated using inputs only.

• Integration with various ecosystem packages like TFMOT, TFLite, TF.js, etc is only supported for inputs and not for tensors in positional and keyword arguments.

*args: Additional positional arguments. May contain tensors, although

this is not recommended, for the reasons above.

**kwargs: Additional keyword arguments. May contain tensors, although

this is not recommended, for the reasons above. The following optional keyword arguments are reserved: - training: Boolean scalar tensor of Python boolean indicating

whether the call is meant for training or inference.

• mask: Boolean input mask. If the layer’s call() method takes a mask argument, its default value will be set to the mask generated for inputs by the previous layer (if input did come from a layer that generated a corresponding mask, i.e. if it came from a Keras layer with masking support).

Returns:

A tensor or list/tuple of tensors.

class d2l.tensorflow.SeqDataLoader(batch_size, num_steps, use_random_iter, max_tokens)

Bases: object

An iterator to load sequence data.

class d2l.tensorflow.Timer

Bases: object

Record multiple running times.

avg()

Return the average time.

cumsum()

Return the accumulated time.

start()

Start the timer.

stop()

Stop the timer and record the time in a list.

sum()

Return the sum of time.

class d2l.tensorflow.TrainCallback(net, train_iter, test_iter, num_epochs, device_name)

Bases: Callback

A callback to visiualize the training progress.

Defined in Section 6.6

on_epoch_begin(epoch, logs=None)

Called at the start of an epoch.

Subclasses should override for any actions to run. This function should only be called during TRAIN mode.

Args:

epoch: Integer, index of epoch. logs: Dict. Currently no data is passed to this argument for this method

but that may change in the future.

on_epoch_end(epoch, logs)

Called at the end of an epoch.

Subclasses should override for any actions to run. This function should only be called during TRAIN mode.

Args:

epoch: Integer, index of epoch. logs: Dict, metric results for this training epoch, and for the

validation epoch if validation is performed. Validation result keys are prefixed with val_. For training epoch, the values of the

Model’s metrics are returned. Example`{‘loss’: 0.2, ‘accuracy’:

0.7}`.

class d2l.tensorflow.TransformerEncoder(*args, **kwargs)

Bases: Encoder

Transformer encoder.

Defined in Section 10.7

call(X, valid_lens, **kwargs)

This is where the layer’s logic lives.

The call() method may not create state (except in its first invocation, wrapping the creation of variables or other resources in tf.init_scope()). It is recommended to create state in __init__(), or the build() method that is called automatically before call() executes the first time.

Args:

inputs: Input tensor, or dict/list/tuple of input tensors.

The first positional inputs argument is subject to special rules: - inputs must be explicitly passed. A layer cannot have zero

arguments, and inputs cannot be provided via the default value of a keyword argument.

• NumPy array or Python scalar values in inputs get cast as tensors.

• Keras mask metadata is only collected from inputs.

• Layers are built (build(input_shape) method) using shape info from inputs only.

• input_spec compatibility is only checked against inputs.

• Mixed precision input casting is only applied to inputs. If a layer has tensor arguments in *args or **kwargs, their casting behavior in mixed precision should be handled manually.

• The SavedModel input specification is generated using inputs only.

• Integration with various ecosystem packages like TFMOT, TFLite, TF.js, etc is only supported for inputs and not for tensors in positional and keyword arguments.

*args: Additional positional arguments. May contain tensors, although

this is not recommended, for the reasons above.

**kwargs: Additional keyword arguments. May contain tensors, although

this is not recommended, for the reasons above. The following optional keyword arguments are reserved: - training: Boolean scalar tensor of Python boolean indicating

whether the call is meant for training or inference.

• mask: Boolean input mask. If the layer’s call() method takes a mask argument, its default value will be set to the mask generated for inputs by the previous layer (if input did come from a layer that generated a corresponding mask, i.e. if it came from a Keras layer with masking support).

Returns:

A tensor or list/tuple of tensors.

class d2l.tensorflow.Updater(params, lr)

Bases: object

For updating parameters using minibatch stochastic gradient descent.

Defined in Section 3.6

class d2l.tensorflow.Vocab(tokens=None, min_freq=0, reserved_tokens=None)

Bases: object

Vocabulary for text.

to_tokens(indices)

property token_freqs

property unk

d2l.tensorflow.accuracy(y_hat, y)

Compute the number of correct predictions.

Defined in Section 3.6

d2l.tensorflow.annotate(text, xy, xytext)

d2l.tensorflow.bbox_to_rect(bbox, color)

Convert bounding box to matplotlib format.

Defined in Section 13.3

d2l.tensorflow.bleu(pred_seq, label_seq, k)

Compute the BLEU.

Defined in Section 9.7.4

d2l.tensorflow.box_center_to_corner(boxes)

Convert from (center, width, height) to (upper-left, lower-right).

Defined in Section 13.3

d2l.tensorflow.box_corner_to_center(boxes)

Convert from (upper-left, lower-right) to (center, width, height).

Defined in Section 13.3

d2l.tensorflow.build_array_nmt(lines, vocab, num_steps)

Transform text sequences of machine translation into minibatches.

Defined in Section 9.5.4

d2l.tensorflow.corr2d(X, K)

Compute 2D cross-correlation.

d2l.tensorflow.count_corpus(tokens)

Count token frequencies.

Defined in Section 8.2

d2l.tensorflow.download(name, cache_dir='../data')

Download a file inserted into DATA_HUB, return the local filename.

Defined in Section 4.10

d2l.tensorflow.download_all()

Download all files in the DATA_HUB.

Defined in Section 4.10

d2l.tensorflow.download_extract(name, folder=None)

Download and extract a zip/tar file.

Defined in Section 4.10

d2l.tensorflow.evaluate_accuracy(net, data_iter)

Compute the accuracy for a model on a dataset.

Defined in Section 3.6

d2l.tensorflow.evaluate_loss(net, data_iter, loss)

Evaluate the loss of a model on the given dataset.

Defined in Section 4.4

d2l.tensorflow.get_data_ch11(batch_size=10, n=1500)

Defined in Section 11.5.2

d2l.tensorflow.get_fashion_mnist_labels(labels)

Return text labels for the Fashion-MNIST dataset.

Defined in Section 3.5

d2l.tensorflow.grad_clipping(grads, theta)

Clip the gradient.

Defined in Section 8.5

d2l.tensorflow.linreg(X, w, b)

The linear regression model.

Defined in Section 3.2

d2l.tensorflow.load_array(data_arrays, batch_size, is_train=True)

Construct a TensorFlow data iterator.

Defined in Section 3.3

d2l.tensorflow.load_corpus_time_machine(max_tokens=-1)

Return token indices and the vocabulary of the time machine dataset.

Defined in Section 8.2

d2l.tensorflow.load_data_fashion_mnist(batch_size, resize=None)

Download the Fashion-MNIST dataset and then load it into memory.

Defined in Section 3.5

d2l.tensorflow.load_data_nmt(batch_size, num_steps, num_examples=600)

Return the iterator and the vocabularies of the translation dataset.

Defined in Section 9.5.4

d2l.tensorflow.load_data_time_machine(batch_size, num_steps, use_random_iter=False, max_tokens=10000)

Return the iterator and the vocabulary of the time machine dataset.

Defined in Section 8.3

d2l.tensorflow.masked_softmax(X, valid_lens)

Perform softmax operation by masking elements on the last axis.

Defined in Section 10.3

d2l.tensorflow.numpy(x, *args, **kwargs)

d2l.tensorflow.plot(X, Y=None, xlabel=None, ylabel=None, legend=None, xlim=None, ylim=None, xscale='linear', yscale='linear', fmts=('-', 'm--', 'g-.', 'r:'), figsize=(3.5, 2.5), axes=None)

Plot data points.

Defined in Section 2.4

d2l.tensorflow.predict_ch3(net, test_iter, n=6)

Predict labels (defined in Chapter 3).

Defined in Section 3.6

d2l.tensorflow.predict_ch8(prefix, num_preds, net, vocab)

Generate new characters following the prefix.

Defined in Section 8.5

d2l.tensorflow.predict_seq2seq(net, src_sentence, src_vocab, tgt_vocab, num_steps, save_attention_weights=False)

Predict for sequence to sequence.

Defined in Section 9.7.4

d2l.tensorflow.preprocess_nmt(text)

Preprocess the English-French dataset.

Defined in Section 9.5

d2l.tensorflow.read_data_nmt()

Load the English-French dataset.

Defined in Section 9.5

d2l.tensorflow.read_time_machine()

Load the time machine dataset into a list of text lines.

Defined in Section 8.2

d2l.tensorflow.seq_data_iter_random(corpus, batch_size, num_steps)

Generate a minibatch of subsequences using random sampling.

Defined in Section 8.3

d2l.tensorflow.seq_data_iter_sequential(corpus, batch_size, num_steps)

Generate a minibatch of subsequences using sequential partitioning.

Defined in Section 8.3

d2l.tensorflow.sequence_mask(X, valid_len, value=0)

Mask irrelevant entries in sequences.

Defined in Section 9.7.2

d2l.tensorflow.set_axes(axes, xlabel, ylabel, xlim, ylim, xscale, yscale, legend)

Set the axes for matplotlib.

Defined in Section 2.4

d2l.tensorflow.set_figsize(figsize=(3.5, 2.5))

Set the figure size for matplotlib.

Defined in Section 2.4

d2l.tensorflow.sgd(params, grads, lr, batch_size)

Minibatch stochastic gradient descent.

Defined in Section 3.2

d2l.tensorflow.show_heatmaps(matrices, xlabel, ylabel, titles=None, figsize=(2.5, 2.5), cmap='Reds')

Show heatmaps of matrices.

Defined in Section 10.1

d2l.tensorflow.show_images(imgs, num_rows, num_cols, titles=None, scale=1.5)

Plot a list of images.

Defined in Section 3.5

d2l.tensorflow.show_list_len_pair_hist(legend, xlabel, ylabel, xlist, ylist)

Plot the histogram for list length pairs.

Defined in Section 9.5

d2l.tensorflow.show_trace_2d(f, results)

Show the trace of 2D variables during optimization.

Defined in Section 11.3.1.1

d2l.tensorflow.size(a)

d2l.tensorflow.squared_loss(y_hat, y)

Squared loss.

Defined in Section 3.2

d2l.tensorflow.synthetic_data(w, b, num_examples)

Generate y = Xw + b + noise.

Defined in Section 3.2

d2l.tensorflow.tokenize(lines, token='word')

Split text lines into word or character tokens.

Defined in Section 8.2

d2l.tensorflow.tokenize_nmt(text, num_examples=None)

Tokenize the English-French dataset.

Defined in Section 9.5

d2l.tensorflow.train_2d(trainer, steps=20, f_grad=None)

Optimize a 2D objective function with a customized trainer.

Defined in Section 11.3.1.1

d2l.tensorflow.train_ch11(trainer_fn, states, hyperparams, data_iter, feature_dim, num_epochs=2)

Defined in Section 11.5.2

d2l.tensorflow.train_ch3(net, train_iter, test_iter, loss, num_epochs, updater)

Train a model (defined in Chapter 3).

Defined in Section 3.6

d2l.tensorflow.train_ch6(net_fn, train_iter, test_iter, num_epochs, lr, device)

Train a model with a GPU (defined in Chapter 6).

Defined in Section 6.6

d2l.tensorflow.train_ch8(net, train_iter, vocab, lr, num_epochs, strategy, use_random_iter=False)

Train a model (defined in Chapter 8).

Defined in Section 8.5

d2l.tensorflow.train_concise_ch11(trainer_fn, hyperparams, data_iter, num_epochs=2)

Defined in Section 11.5.2

d2l.tensorflow.train_epoch_ch3(net, train_iter, loss, updater)

The training loop defined in Chapter 3.

Defined in Section 3.6

d2l.tensorflow.train_epoch_ch8(net, train_iter, loss, updater, use_random_iter)

Train a model within one epoch (defined in Chapter 8).

Defined in Section 8.5

d2l.tensorflow.train_seq2seq(net, data_iter, lr, num_epochs, tgt_vocab, device)

Train a model for sequence to sequence.

Defined in Section 9.7.2

d2l.tensorflow.transpose_output(X, num_heads)

Reverse the operation of transpose_qkv.

Defined in Section 10.5

d2l.tensorflow.transpose_qkv(X, num_heads)

Transposition for parallel computation of multiple attention heads.

Defined in Section 10.5

d2l.tensorflow.truncate_pad(line, num_steps, padding_token)

Truncate or pad sequences.

Defined in Section 9.5

d2l.tensorflow.try_all_gpus()

Return all available GPUs, or [cpu(),] if no GPU exists.

Defined in Section 5.6

d2l.tensorflow.try_gpu(i=0)

Return gpu(i) if exists, otherwise return cpu().

Defined in Section 5.6

d2l.tensorflow.update_D(X, Z, net_D, net_G, loss, optimizer_D)

Update discriminator.

Defined in Section 17.1

d2l.tensorflow.update_G(Z, net_D, net_G, loss, optimizer_G)

Update generator.

Defined in Section 17.1

d2l.tensorflow.use_svg_display()

Use the svg format to display a plot in Jupyter.

Defined in Section 2.4