Package: pysparklyr 0.2.1
pysparklyr: Provides a 'PySpark' Back-End for the 'sparklyr' Package
It enables 'sparklyr' to integrate with 'Spark Connect', and 'Databricks Connect' by providing a wrapper over the 'PySpark' 'python' library.
Authors:
pysparklyr_0.2.1.tar.gz
pysparklyr_0.2.1.zip(r-4.7)pysparklyr_0.2.1.zip(r-4.6)pysparklyr_0.2.1.zip(r-4.5)
pysparklyr_0.2.1.tgz(r-4.6-any)pysparklyr_0.2.1.tgz(r-4.5-any)
pysparklyr_0.2.1.tar.gz(r-4.7-any)pysparklyr_0.2.1.tar.gz(r-4.6-any)
pysparklyr_0.2.1.tgz(r-4.6-emscripten)
manual.pdf |manual.html✨
card.svg |card.png
pysparklyr/json (API)
NEWS
| # Install 'pysparklyr' in R: |
| install.packages('pysparklyr', repos = c('https://mlverse.r-universe.dev', 'https://cloud.r-project.org')) |
Bug tracker:https://github.com/mlverse/pysparklyr/issues
databrickspysparksparkspark-connect
Last updated from:fc552da55c. Checks:7 WARNING, 2 OK. Indexed: yes.
| Target | Result | Time | Files | Syslog |
|---|---|---|---|---|
| linux-devel-x86_64 | WARNING | 169 | ||
| source / vignettes | OK | 220 | ||
| linux-release-x86_64 | WARNING | 180 | ||
| macos-release-arm64 | WARNING | 106 | ||
| macos-oldrel-arm64 | WARNING | 133 | ||
| windows-devel | WARNING | 135 | ||
| windows-release | WARNING | 119 | ||
| windows-oldrel | WARNING | 128 | ||
| wasm-release | OK | 152 |
Exports:connection_databricks_shinyappdeploy_databricksinstall_databricksinstall_pysparkinstalled_componentsml_prepare_datasetpyspark_configrequirements_writespark_connect_service_startspark_connect_service_stop
Dependencies:arrowaskpassassertthatbitbit64blobclicodetoolsconfigconnectcredscpp11curlDBIdbplyrdplyrfsgenericsglobalsglueherehttrhttr2jsonlitelatticelifecyclemagrittrMatrixmimeopensslpillarpkgconfigpngprocessxpspurrrR6rappdirsRcppRcppTOMLreticulaterlangrprojrootrstudioapisparklyrstringistringrsystibbletidyrtidyselectutf8uuidvctrswithrxml2yaml
Readme and manuals
Help Manual
| Help page | Topics |
|---|---|
| Deploys Databricks backed content to publishing server | deploy_databricks |
| Installs PySpark and Python dependencies | install_databricks install_pyspark |
| Lists installed Python libraries | installed_components |
| Creates the 'label' and 'features' columns | ml_prepare_dataset |
| Read Spark configuration | pyspark_config |
| Writes the 'requirements.txt' file, containing the needed Python libraries | requirements_write |
| Starts and stops Spark Connect locally | spark_connect_service_start spark_connect_service_stop |
