Alluxio

Alluxio

Alluxio, data orchestration for analytics and machine learning in the cloud
Under Apache License 2.0
By Alluxio

tensorflow data-analysis spark hadoop alluxio memory-speed presto data-orchestration virtual-distributed-filesystem



What is Alluxio

Alluxio (formerly known as Tachyon)
is a virtual distributed storage system. It bridges the gap between
computation frameworks and storage systems, enabling computation applications to connect to
numerous storage systems through a common interface. Read more about
Alluxio Overview.


The Alluxio project originated from a research project called Tachyon at AMPLab, UC Berkeley,
which was the data layer of the Berkeley Data Analytics Stack (BDAS).
For more details, please refer to Haoyuan Li's PhD dissertation
Alluxio: A Virtual Distributed File System.


Who Uses Alluxio

Alluxio is used in production to manage Petabytes of data in many leading companies, with
the largest deployment exceeding 3,000 nodes. You can find more use cases at
Powered by Alluxio or visit our first community conference (Data Orchestration Summit) to learn from other community members!


Who Owns and Manages Alluxio Project

Alluxio Open Source Foundation is the owner of Alluxio project.
Project operation is done by Alluxio Project Management Committee (PMC).
You can checkout more details in its structure and how to join Alluxio PMC
here.


Community and Events

Please use the following to reach members of the community:



Download Alluxio
Binary download

Prebuilt binaries are available to download at https://www.alluxio.io/download .


Docker

Download and start an Alluxio master and a worker. More details can be found in documentation.


```console


Create a network for connecting Alluxio containers

$ docker network create alluxio_nw


Create a volume for storing ufs data

$ docker volume create ufs


Launch the Alluxio master

$ docker run -d --net=alluxio_nw \
-p 19999:19999 \
--name=alluxio-master \
-v ufs:/opt/alluxio/underFSStorage \
alluxio/alluxio master


Launch the Alluxio worker

$ export ALLUXIO_WORKER_RAMDISK_SIZE=1G
$ docker run -d --net=alluxio_nw \
--shm-size=${ALLUXIO_WORKER_RAMDISK_SIZE} \
--name=alluxio-worker \
-v ufs:/opt/alluxio/underFSStorage \
-e ALLUXIO_JAVA_OPTS="-Dalluxio.worker.ramdisk.size=${ALLUXIO_WORKER_RAMDISK_SIZE} -Dalluxio.master.hostname=alluxio-master" \
alluxio/alluxio worker
```


MacOS Homebrew

console
$ brew install alluxio


Quick Start

Please follow the Guide to Get Started
to run a simple example with Alluxio.


Report a Bug

To report bugs, suggest improvements, or create new feature requests, please open a Github Issue.
If you are not sure whether you run into bugs or simply have general questions with respect to Alluxio, post your questions on Alluxio Slack channel.


Depend on Alluxio

Alluxio project provides several different client artifacts for external projects to depend on Alluxio client:



Here are examples to declare the dependecies on alluxio-shaded-client using Maven:


xml
<dependency>
<groupId>org.alluxio</groupId>
<artifactId>alluxio-shaded-client</artifactId>
<version>2.6.0</version>
</dependency>


Contributing

Contributions via GitHub pull requests are gladly accepted from their original author. Along with
any pull requests, please state that the contribution is your original work and that you license the
work to the project under the project's open source license. Whether or not you state this
explicitly, by submitting any copyrighted material via pull request, email, or other means you agree
to license the material under the project's open source license and warrant that you have the legal
authority to do so.
For a more detailed step-by-step guide, please read
how to contribute to Alluxio.
For new contributor, please take two new contributor tasks.


For advanced feature requests and contributions,
Alluxio core team is hosting regular online meetings with community users and developers to iterate the project in two special interest groups:



Subscribe our public calendar to join us.


Useful Links