Testing multi machine hadoop cluster
So I was testing setup of a hadoop cluster and running map reduce. There are preliminary notes in first post. One key, I ran all commands under openssh as that user Setup the user Main machine: sudo usermod --shell /bin/bash mainhdfs Start openssh openssh su - mainhdfs sudo apt-get install openssh-server sudo systemctl enable ssh sudo systemctl enable ssh --now sudo systemctl start ssh su - mainhdfs sudo mkdir /usr/local/hadoop sudo mv hadoop-3.3.6 /usr/local/hadoop/ sudo chown own -R mainhdfs:hadoop /usr/local/hadoop sudo chown -R mainhdfs:hadoop /usr/local/hadoop Setup more on ssh: ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys chmod 600 ~/.ssh/authorized_keys chmod 700 ~/.ssh Write the following .bashrc and .bash_profile for that user export HADOOP_HOME=/usr/local/hadoop/hadoop-3.3.6 export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin export HADOOP_CONF_...