Posts

Showing posts from August, 2024

Testing multi machine hadoop cluster

 So I was testing setup of a hadoop cluster and running map reduce. There are preliminary notes in first post. One key, I ran all commands under openssh as that user Setup the user Main machine: sudo usermod --shell /bin/bash mainhdfs Start openssh   openssh   su - mainhdfs   sudo apt-get install openssh-server   sudo systemctl enable ssh   sudo systemctl enable ssh --now   sudo systemctl start ssh   su - mainhdfs   sudo mkdir /usr/local/hadoop   sudo mv hadoop-3.3.6 /usr/local/hadoop/   sudo chown own -R mainhdfs:hadoop /usr/local/hadoop   sudo chown -R mainhdfs:hadoop /usr/local/hadoop Setup more on ssh: ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys chmod 600 ~/.ssh/authorized_keys chmod 700 ~/.ssh Write the following .bashrc and .bash_profile for that user export HADOOP_HOME=/usr/local/hadoop/hadoop-3.3.6 export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop export JAVA_HOME=/usr/l

My post on #Raygun

#Raygun thank you for failing.  It is OK to fail.  Fail, move on.  Our own President Biden failed recently.  Shane Gillis failed on SNL.  George Santos failed.  Ingrid Andress on national anthem.  Doing good is better, but OK to fail.  Who else? 

More posts around deep learning and google deep mind

 robotics http://hutter1.net/ai/suaibook.pdf https://arxiv.org/abs/1708.04782 https://github.com/google-deepmind/pysc2/blob/master/pysc2/lib/named_array.py https://github.com/google-deepmind/deepmind-research/tree/master/tandem_dqn https://www.youtube.com/watch?v=hCeJeq8U0lo&t=9s https://arxiv.org/pdf/2105.14039 https://arxiv.org/pdf/2102.05182 https://openreview.net/forum?id=nPHA8fGicZk https://github.com/google-deepmind/deepmind-research/tree/master/tandem_dqn . Neuromorphic Computing Technologies: Neuromorphic chips like Intel's Loihi or IBM's TrueNorth are designed to mimic the structure and function of the human brain, making them ideal for brain-like AI models. Purpose: These chips are designed to support spiking neural networks (SNNs), which are more biologically plausible than traditional artificial neural networks. 2. Spiking Neural Networks (SNNs) Algorithms: SNNs use spikes (events) to transmit information, similar to how neurons communicate in the brain. This al

Was going to watch twitter x - Trump and Elon Musk - It crashed

 Was going to watch twitter x - Trump and Elon Musk - It crashed

Random thoughts on AI

 Random AI Thoughts Posts - things I am looking at: scale ai Pinecone Perplexity Notion Hebbia Character AI --- web server web client stand alone -    limited html and http - validates your html and shit   very strict     -- tomcat based      -- my own ML-From-Scratch Description: This repository provides implementations of various machine learning algorithms from scratch in Python, without using libraries like Scikit-learn. It's educational and shows how algorithms like k-means clustering, decision trees, and linear regression work under the hood. Repository: ML-From-Scratch on GitHub You-Get Description: A simple and powerful tool for downloading media files from the web. It uses machine learning for analyzing and categorizing the downloaded media. Features: Supports various media sites, custom download options, and automated media organization. 3. OpenAI Gym Description: A toolkit for developing and comparing reinforcement learning algorithms. While not an end-user application,

Lilou vs Hong 10

 This is a good competition lilou vs hong 10 #raygun https://www.youtube.com/watch?v=JKQ83Bo9g3s

Random thoughts and posts

Image
 

See older post on cellular automata

 An older post on cellular automata https://berlinbrowndev.blogspot.com/2013/02/wolframs-cellular-automata-new-kind-of.html A New Kind of Science The name Stephan Wolfram has been mentioned several times in this post. He is the founder of Wolfram|Research, his company is known for the popular Mathematica software suite and Wolfram|Alpha knowledge engine. He did not initially discover cellular automata but recently he has been a prominent figure in its advocacy. He spent 10 years working on his book, A New Kind of Science. In the 1300 page tome, he discusses how cellular automata can be applied to every field of science from biology to physics. NKA is a detailed study of cellular automata programs. Basic Cellular Automata Figure: Wolfram's Elementary CA Rule 30. Look at 3 bit input and 1 bit output. The diagram above depicts the rule 30 program (or rule 30 elementary cellular automaton). There are 8 input states (2 ^ 3) and an output state of one or zero. If you look at the diagram

What we learned from machine learning

What we learned machine learning and deep learning So we have learned a lot from the machine learning and deep learning from Sebastian Raschka. See his blog https://magazine.sebastianraschka.com/ And the following machine learning and deep learning courses. https://sebastianraschka.com/blog/2021/ml-course.html https://sebastianraschka.com/blog/2021/dl-course.html The first set of courses cover: kNN - k nearest neighbor decision trees and popular ML python libraries.

Basic install jupyter

 Jupyter is the new thing for data science colab.  It is a web based system for sharing interactive documents. To install linux (mint): Here I basically installed via pip with Linux sudo apt install jupyter-core    sudo  apt --fix-broken install jupyter-core    sudo apt install jupyter-core    pip install jupyterlab    python3 -m jupyterlab And then with Mac Sonoma I had python3 already installed https://jupyter.org/install This one worked with Mac as well with python3 and myenv With the virtual environment setup   jupyter notebook   mkdir ~/myenv   python3 -m venv ~/myenv   ~/myenv/bin/python   ~/myenv/bin/pip install    ~/myenv/bin/pip install numpy   ~/myenv/bin/python -m jupyter notebook   ~/myenv/bin/pip install jupyter   ~/myenv/bin/pip install --upgrade pip   ~/myenv/bin/python -m jupyter notebook   ~/myenv/bin/pip install numpy pandas numpy scipy matplotlib scikit-learn   ~/myenv/bin/python -m jupyter notebook