Fixed torch_scalar_tensor() with requires_grad = TRUE, which returned a
non-leaf tensor because $squeeze() was applied after requires_grad was
set, so .grad was never populated after backward(). (#1368)
Fixed nn_multihead_attention() and nnf_multi_head_attention_forward() with bias = FALSE
when query differs from key (i.e. cross- and encoder-decoder attention), which failed with
object 'k' not found. The key and value projections were only computed when a bias was present.
Fixed lr_one_cycle() with anneal_strategy = "linear", which failed with
attempt to apply non-function because the annealing function was never assigned.
Multi-worker dataloaders now use POSIX shared memory for tensor transfer on
Unix systems, resulting in up to 2x faster data loading. To revert to the
previous behavior, set options(torch.dataloader_use_shm = FALSE). (#1456)
On Windows, the install lib directory is now prepended to PATH at load so
cuDNN's lazily-loaded sub-DLLs (e.g. cudnn_graph64_9.dll) resolve; cuDNN-backed
CUDA ops previously failed with "Could not locate cudnn_graph64_9.dll".
The ignore_index argument of nnf_cross_entropy(), nnf_nll_loss(),
nn_cross_entropy_loss() and nn_nll_loss() is now interpreted as the 1-based class
index that target already used. It was forwarded to libtorch unconverted while the
target was converted, so it ignored the class to the left of the requested one, and the
last class could not be ignored at all. Passing 0 is now an error instead of silently
ignoring the first class. The default -100, and any other negative sentinel, is
unaffected. The documented target range was corrected to \eqn{1 \leq target \leq C}.
torch_triu_indices() and torch_tril_indices() now return 1-based indexes (#1382).$indices() now returns 1-based indexes (#1382).torch_sitrep() for torch installation situation report (#1415, @cregouby).%*% operator for torch tensors (#1379, @caio-hamamura).head() and tail() methods for torch tensors (#1388).torch_scaled_dot_product_attention() (#1404).torch_sparse_sampled_addmm() (#1427).torch_ldexp() (#1407).cudatoolkit R package for CUDA library loading (#1422).as_array() support for sparse tensors (#1387).to_sparse() crash, and as_array() detection (#1431).dataset()/nn_module() crash when initialize parameter is named d (#1424, @Chandraveer-Singh).bfloat16 not being recognized as a floating point dtype (#1408).lantern.h include causing macro redefinition warning (#1430).Rf_findVarInFrame with R_getVarEx for R 4.6 compatibility (#1421).nn_aum_loss() (#1310 @cregouby)Bug fixes:
RTLD_DEEPBIND (#1268)optim_ignite_<name>()) are available: Adam, AdamW, Adagrad, RMSprop,SGD.
These can be used as drop-in replacements for optim_<name> but are considerably
faster as they wrap the LibTorch implementation of the optimizer.
The biggest speed differences can be observed for complex optimizers such as AdamW.torch_iinfo() now support all integer dtypes (#1190 @cregouby)nnf_multi_head_attention_forward() (#1205)jit_trace (#1246)$finalize_deep_clone() method for nn_module which
allows to run some code after cloning a module.compare_proxy method for the torch_tensor type was added
it allows to compare torch tensors using testthat::expect_equal().nnf_multi_head_attention_forward (#1154 @cregouby)$detach() on a tensor now preserves attributes (#1136)create_nn_module_callable()grad_fn now appends a new line at the end.local_autocast(). (#1142)torch_arange() causing it to return 1:(n-1) values when specific request dtype = torch_int64() (#1160)torch_save serialization format. It's ~10x faster and since it's based on safetensors, files can be read with any safetensors implementation. (#1071)torch_load no longer supports device=NULL to load weights in the same device they were saved. (#1085)as.array now moves tensors to the cpu before copying data into R. (#1080)dtype with a NULL. (#1090)torch_cfloat64. (#1091)out_features attribute in the nn_linear module. (#1097)nn_modules with empty state dicts. (#1108)distr_multivariate_normal now correctly handles precision matrix's. (#1110)length.torch_tensor implementation to R7 to avoid problems when a torch dataset has the torch_tensor class. (#1111)nn_module. (#1123)load_state_dict() for optimizers now default to cloning the tensors in the state dict, so they don't keep references to objects in the dict. (#1041)nn_utils_weight_norm (#1025)jit_ops allowing to access JIT operators. (#1023)with_device and local_device to allow temporarily modify the default device tensors get initialized. (#1034)nnf_gelu() and nn_gelu() gained the approximate argument. (#1043)!= for torch devices. (#1042)nn_module_dict(). (#1046)load_state_dict(), also using less memory. It' possible to use the legacy implementation if required, see PR. (#1051)as.numeric(). (#1056)torch_tensor_from_buffer() and buffer_from_torch_tensor() that allow low level creation of torch tensors. (#1061, #1062)install_torch(). (#1055)state_dict() even if they should not. (#1036)optim_adamw class name.nn_cross_entropy_loss class name. (#1043)RemoteSha is a package version. (#1058)CXXSTD to Makevars to enable C+11 compilation options..refer_to_state_dict to the load_state_dict() nn_module() method. Allows loading the state dict into the model keeping parmaters as references to that state dict. (#1036)nn_silu() and nnf_silu(). (#985)nn_modules. (#986)local_no_grad() and local_enable_grad() as alternatives for the with_ functions. (#990)optim_adamw optimizer. (#991)batch_size=1 is used. (#994)torch_where now returns 1-based indices when it's called with the condition argument only. (#951, @skeydan)torch_fft_fftfreq() function is now exported. (#950, @skeydan)distr_normal$sample() not being able to generate reproducible results after setting seeds. (#938)torch_cat error message now correctly reflects 1-based indexing. (#952, @skeydan)sprintf. (#959, @shikokuchuo)glue (#960)torch_bincount is now 1-based indexed. (#896)torch_movedim() and $movedim() are now both 1-based indexed. (#905)cuda_synchronize() to allow synchronization of CUDA operations. (#887)cuda_empty_cache() to allow freeing memory from the caching allocator to the system. (#903)$is_sparse() method to check wether a Tensor is sparse or not. (#903)dataset_subset now adds a class to the modified dataset that is the same as the original dataset classes postfixed with _subset. (#904)torch_serialize() to allow creating a raw vector from torch objects. (#908)torch_arange that was causing the end value not getting included in the result. (#885, @skeydan)install_torch(reinstall = TRUE). (#883)dims argument in torch_tile() is no longer modified, as it's not meant to be the a 1-based dimension. (#905)nn_module$state_diict() now detaches output tensors by default. (#916)$ method for R7 classes in C/C++ to improve speed when calling methods. (#873)backward() call. This improves speed because we no longer need to call GC
everytime backward is called. (#873)nnf_ functions have been updated to use a single torch_ kernel instead of the custom implementation. (#896)torch_bernoulli() with device="gpu". (#906)PRECXX11ABI=1. This can be useful in environments with older versions of GLIBC. (#870)options(torch.serialization_version = 1) if you want your file to be readable by older versions. (#803)linalg_matrix_rank and linalg_pinv gained atol and rtol arguments while deprecating tol and rcond. (#835)keep_graph now defaults to the value of create_graph when calling $backward(). We also renamed it to retain_graph to match PyTorch. (#811)optimizer now carry the classname in the generator and in instances. Optimizer generators now have the class torch_optimizer_generator. The class of torch optimizers has been renamed from torch_Optimizer to torch_optimizer. (#814)nn_prune_head() to prune top layer(s) of a network (#819 @cregouby)torch_kron() is now exported (#818).nn_embedding_bag. (#827, @egillax)nn_multihead_attention now supports the batch_first option. (#828, @jonthegeek)x$grad <- new_grad. (#832)sampler() is now exported allowing to create custom samplers that can be passed to dataloader(). (#833)nn_modules without a initialize method is now supported. (#834)lr_reduce_on_plateau learning rate scheduler. (#836, @egillax)torch_tensor(NULL) no longer fails. It now returns a tensor with no dimensions and no data. (#839)nn_l1_loss. (#825, @sebffischer)embed_dim is not divisible by num_heads in nn_multihead_attention. (#828)cuda_runtime_version() to query the CUDA Tolkit version that torch is using. (#790)torch_sort and Tensor$sort now return 1-indexed results. (#709, @mohamed-180)dim from 1 to 2 in nnf_cosine_similarity. (#769)jit_save_for_mobile allows to save a traced model in bytecode form, to be loaded by a LiteModuleLoader. (#713)is_torch_tensor to check wether an object is a tensor or not. (#730, @rdinnager)cuda_get_device_properties(device) that allows one to query device capability and other properties. (#734, @rdinnager)call_torch_function() to allow calling potentially unexported torch core functions. (#743, @rdinnager)inst directory. This will allow for packages extending torch to bind directly to its C++ library. (#718)dataset_subset will use the .getbatch method of the wrapped dataset if one is available. (#742, @egillax)nn_flatten and nn_unflatten modules. (#773)cuda_memory_stats() and cuda_memory_summary() to verify the amount of memory torch is using from the GPU. (#774)backends_cudnn_version() to query the CuDNN version found by torch. (#774).validate_sample for the Distribution class that would incorrectly check for tensors. (#739, @hsbadr)autograd_functions. (#750)autograd_grad to deadlock when used with custom autograd functions. (#771)torch_max and torch_min that would fail with length=2 Tensors. (#772)torch_tensor is now a C++ only function slighly increasing performance in a few situations. (#784)nn_sequential is now a bare nn_module, allowing to easily inherit from it. This is a breaking change if you used the name argument. The name behavior can be achieved by subclassing; see the tests in the PR. (#699)nn_sequential modules using [. (#678, @mohamed-180)padding='same' and padding='valid' when using convolutions. (#679)nnf_cross_entropy now uses the ATen cross_entropy operation directly instead of doing logsoftmax + NLLLoss. (#680)nn_sequential and still want that the specific S3 methods still work. (#701)torch_manual_seed() now matches PyTorch's behavior so we can more easily compare implementations. Since this is a breaking change we added the torch.old_seed_behavior=TRUE option so users can stick to the old behavior. (#639)torch_index or torch_index_put. (#649)nn_modules. (#593)jit_compile function that allows compiling arbitrary TorchScript code into script function that can be serialized and executed. (#601)jit_trace support for nn_module created from R. (#604)contrib_sort_vertices to efficiently sort vertices on CUDA. (#619)with_detect_anomaly to debug autograd errors. (#628)traced_module$graph_for() to allow inspecting the optimized jit graph. (#643)slc to allow dynamically creating slices when indexing tensors. (#648).getbatch method that didn't return a torch_tensor. (#615)%/% caused by a call to deprecated torch_floor_divide (#616)gc when CUDA memory is exhausted we now call R_RunPendingFinalizers. This should improve memory usage, because we will now delete tensors earlier. (#654)torch_multinomial now returns 1-based indexes to comply with 1-based indexing across torch. (#588)jit_trace functions with more than 1 argument. (#544)torch_diff function and added docs for it. (#565)device argument to torch_load() allowing one to select to which device parameters should be loaded. (#578)distr_categorical() (#576)distr_mixture_same_family() (#576)load_state_dict() and state_dict() for optimizers. (#585)lists containing torch_tensors using torch_save. This allows us to save the state of optimizers and modules using torch_save(). (#586)nn_multihead_attention when q,k,v inputs not all the same. (@jonathanbratt #540)$copy_ so it correctly respects the src requires_grad() when reloading saved models with torch_load(). (#545)nn_init_xavier_normal_() and nn_init_xavier_uniform_() standard deviation calculation. (#557)torch_tensordot() when called when infering dimensions. (#563).getbatch now takes an integer vector as input instead of a list(). (#572)tensor$size() when indexing with negative numbers. (#570)log_prob of distr_bernoulli() (#581)XPtrTorchOptionalTensor class. (#565)torch_nonzero and tensor$nonzero() now return 1-based indexes. (#432)torch_arange returns in the closed interval [start, end] instead of the half open [start, end). This makes it behave similar to R's seq. (#506)torch_split now accepts a list of sizes as well as a fixed size. (#429)nn_layer_norm. (#435)timeout=360 as install_torch() parameter for large file download (@cregouby #438)install_torch_from_file() and get_install_libs_url()for setup cases where direct download is not possible (@cregouby #439)mean.torch_tensor (#448)worker_globals and worker_packages allowing to easily pass objects to workers in parallel dataloaders (#449).nn_group_norm and fixed a bug in nnf_group_norm (#474)as.list for nn_modules (#492)nn_lstm, nn_gru and nn_gru can now use cudnn accelerations when available (#503).!= for torch dtypes. (#524)optimizer function allowing to easily implement custom optimizers. (#527)optim_lbfgs that would make model objects exponentially big. (#431)NaNs in L-BFGS optimizer (#433)torch_normal. (#450)torch_save. (#454)nnf_pad when using mode='circular'. (#471)nn_multihead_attention (#496)nn_lstm (#500)to method of nn_module that would reset the requires_grad attribute of parameters. (#501)strong_wolfe option to optim_lbfgs. (#517)nn_init_trunc_normal_ initializer function. (#535)torch_dtype() being an R6 class. (#451)torch_one_hot and nnf_one_hot use 1-based indexing. (#410)nn_module$eval() and nn_module$train() now return a callable nn_module instead of a nn_Module. (#425)gc when torch might need more memory (#402)nn_module (#417)requires_grad attribute using the $<- operator (#419)length method for the nn_sequential container. (#423)nnf_normalize and x$norm() as they were not able to be called (#409)nn_module documentation (#399).coro::exhausted intead of raising stop_iteration_error when the dataloader exceeds. (#366)torch_argsort and x$argsort to return 1-based indexes (#342)torch_argmax, torch_argmin, x$argmax() and x$argmin() return 1-based indexes. (#389)$element_size() method (@dirkschumacher #322)$bool() method (@dirkschumacher #323)torch__addr and torch__addr_ have been removed as they are no longer available in LibTorch 1.7.long tensors to R and there's a chance of an integer overflow. (#347)private and active methods in nn_module's and dataset's. (#349)nn_batch_norm3d (@mattwarkentin #354)nn_lstm and nn_gru modules. (#362)nn_module UI by improving autocomplete support and adding a print method (#391)grad_fn of a Tensor that doesn't have one.
See (#321)torch_norm so it can be called with a dim argument. (#345)torch_hann_window with an invalid NULL window_length. (#351)torch_stft calls for LibTorch 1.7 (added the return_complex argument) (#355)nvcc --version instead of nvidia-smi to find the CUDA version as nvidia-smi reports the latest supported version and not the installed one. (#363)RandomSampler(replacement = TRUE) to never take the last
element in the dataset. (84861fa)torch_topk and x$topk so the returned indexes are 1-based (#280)1 - torch_tensor(1, device = "cuda") to
fail because 1 was created in the CPU. (#279)dataloader output (#286)torch_narrow, Tensor$narrow() and Tensor$narrow_copy are now indexed
starting at 1. (#294)Tensor$is_leaf is now an active method. (#295)torch_einsum. (#296)nn_module_list() to correctly name added modules, otherwise they are not
returned when doing state_dict() on it. (#300)nn_batchnorm* so it returns the same results as PyTorch (#302)nn_module$parameter when there were shared parameters
between layers. (#306)$max and $min to return 1-based indexes. (#315)utils_data_default_collate to support converting R objects to
torch tensors when needed. (#269)as.matrix method for torch Tensors. (#282)print(totrch_tensor(1:40)) if it
spans for more than 30 lines. This is useful for not spamming the console or
taking very long to print when you print a very large tensor. (#283)nnf_contrib_sparsemax and nn_contrib_sparsemax. (#309)$<- and [[<- operators for the nn_module class. (#253)nn_parameter, nn_buffer, and is_* auxiliary functions.NEWS.md file to track changes to the package.