Decision Trees with r project on Linux

Chaid Trees with R Project on Linux

Start R and install Partykit:
$ R
> install.packages("partykit")

On a command line download the chaid sources:
svn checkout svn://svn.r-forge.r-project.org/svnroot/chaid/

$ cd chaid
$ R CMD INSTALL pkg/

On R:
> library("CHAID")
Loading required package: partykit
Loading required package: grid



Chaid project:
http://r-forge.r-project.org/scm/?group_id=343

How to obtain the flash videos in Firefox 4

Flash videos were stored in /tmp/FlashXXXX and i was able to copy them to another place like cp /tmp/FlashXXX ~/Desktop/new.video.mpeg.

Since Firefox 4 i thought for a moment that everything changed and my happy days were over but i was wrong, i remembered that with Linux there are no sad days.

How to obtain the flash videos with Firefox 4:

When the video already finished loading in the browser (being Youtube or any other), do this:

ps -ef | grep libflashplayer.so | grep -v grep

walter   20040 19812  9 11:52 ?        00:53:34 /usr/lib/firefox-3.6.17/plugin-container /home/walter/.mozilla/plugins/libflashplayer.so 19812 plugin true

Now, with the PID, inspect what it is opening:


root@talento:~# lsof -p 20040 | grep Flash
lsof: WARNING: can't stat() fuse.gvfs-fuse-daemon file system /home/walter/.gvfs
      Output information may be incomplete.
plugin-co 20040 walter   16u   REG        8,2  1249930 21708868 /tmp/FlashXXQANNm2 (deleted)
root@talento:~#

You are interested in the file descriptor id, these are the file descriptors from that process:


root@talento:~# ls -la /proc/20040/fd/
total 0
dr-x------ 2 walter walter  0 2011-04-30 21:12 .
dr-xr-xr-x 7 walter walter  0 2011-04-30 21:06 ..
lr-x------ 1 walter walter 64 2011-04-30 21:12 0 -> /dev/null
lrwx------ 1 walter walter 64 2011-04-30 21:12 1 -> /home/walter/.xsession-errors
l-wx------ 1 walter walter 64 2011-04-30 21:12 10 -> pipe:[18679239]
lr-x------ 1 walter walter 64 2011-04-30 21:12 11 -> pipe:[18679240]
l-wx------ 1 walter walter 64 2011-04-30 21:12 12 -> pipe:[18679240]
lrwx------ 1 walter walter 64 2011-04-30 21:12 13 -> socket:[18679243]
lr-x------ 1 walter walter 64 2011-04-30 21:12 14 -> /home/walter/.mozilla/firefox/2te1hj2x.default/cert8.db
lr-x------ 1 walter walter 64 2011-04-30 21:12 15 -> /home/walter/.mozilla/firefox/2te1hj2x.default/key3.db
lrwx------ 1 walter walter 64 2011-04-30 21:15 16 -> /tmp/FlashXXQANNm2 (deleted)
lr-x------ 1 walter walter 64 2011-04-30 21:16 17 -> pipe:[19890054]
l-wx------ 1 walter walter 64 2011-04-30 21:16 18 -> pipe:[19890054]
lr-x------ 1 walter walter 64 2011-04-30 21:16 19 -> pipe:[19890055]
lrwx------ 1 walter walter 64 2011-04-30 21:12 2 -> /home/walter/.xsession-errors
l-wx------ 1 walter walter 64 2011-04-30 21:16 20 -> pipe:[19890055]
lrwx------ 1 walter walter 64 2011-04-30 21:16 21 -> socket:[19890059]
lrwx------ 1 walter walter 64 2011-04-30 21:12 22 -> socket:[19338003]
lr-x------ 1 walter walter 64 2011-04-30 21:12 23 -> anon_inode:inotify
lrwx------ 1 walter walter 64 2011-04-30 21:12 3 -> socket:[18679123]
lrwx------ 1 walter walter 64 2011-04-30 21:12 4 -> anon_inode:[eventpoll]
lrwx------ 1 walter walter 64 2011-04-30 21:12 42 -> socket:[18679124]
lrwx------ 1 walter walter 64 2011-04-30 21:12 5 -> socket:[18679236]
lrwx------ 1 walter walter 64 2011-04-30 21:12 6 -> socket:[18679237]
lr-x------ 1 walter walter 64 2011-04-30 21:12 7 -> pipe:[18679238]
l-wx------ 1 walter walter 64 2011-04-30 21:12 8 -> pipe:[18679238]
lr-x------ 1 walter walter 64 2011-04-30 21:12 9 -> pipe:[18679239]

What is of our interest if the Flash process, displayed in the previous lines.

Now, you can copy the file to a safe place, because, if you close Firefox, those files are released and lost.

cp /proc/20040/fd/16 ~/Desktop/A.Movie.mpeg

Now if you want to put everything in a script:

root@talento:~# cat saveflash.sh
#!/bin/bash

PID=`ps -ef | grep libflashplayer.so | grep -v grep | awk '{print $2}'`
FD=`lsof -p $PID | grep Flash | awk '{print $4}' | sed 's/u$//'`
cp /proc/$PID/fd/$FD "$1"

And remember, you can "invite me a coffee" :)

Yahoo KDD Cup

In a few days starts the Yahoo KDD Cup, i'll participate in this one.  It is an interesting task where the rate of users has to be predicted.  The prediction task is based on previous users behaviour, finding patterns and common music/artists/genre and profiling new customers based on the music they do like.  I hope i have fun during this competition and keep learning from these experiences.

If possible, i would keep doing all my mining/machine learning scripts in perl, so i will share the solutions for the problems i had to face during this experience.

I wanted to search an id inside an array, to do this i did:

open TODO, "todo" or die $!;
my @todo = ;
close TODO;
@array = ();
foreach (@todo) {
   my ($id, $type) = split(/\t/);
   chop ($type);
   $array[$id] = $type;
}

my ($id, $rank, $day, $time) = split(/\t/, $linea);

Installing R project for statistics in Fedora Linux.

Is is as easy as this:

[root@dyn8382 ~]# yum install R R-devel
[root@dyn8382 ~]# R
>install.packages("arules", repos="http://R-Forge.R-project.org")




http://www.r-project.org/

Datamining with Linux

I'm currently doing the Datamining Specialist course and i try to use open source solutions, and if not open source, linux solutions... because i am a unix user.

This is a list of software i recommend to try, i am studying some features and they are
all great:

WEKA (http://www.cs.waikato.ac.nz/ml/weka/)

Weka is a collection of machine learning algorithms for data mining tasks. The algorithms can either be applied directly to a dataset or called from your own Java code. Weka contains tools for data pre-processing, classification, regression, clustering, association rules, and visualization. It is also well-suited for developing new machine learning schemes.

The R Project for Statistical Computing (http://www.r-project.org/)
R is a free software environment for statistical computing and graphics.

Matlab (http://www.mathworks.com/)
This is a commercial software that can be used to create models of artificial neural networks and any kind of math problems.

dTree ( http://www.aispace.org/dTree/)
This tool demonstrates how to build a decision tree using a training data set and then use the tree to classify unseen examples in a test data set.

If you know some other soft, please post it and i'll try it.

VMplayer crash in fedora 12

After installing VMplayer from the bundle, the 3.0.1 version i were surprised because it crashed just one second after running it.  Thanks that another co-blogger posted the solution, too simple in fact:

cd /usr/lib/vmware/lib

mv libcurl.so.4 libcurl.so.4.old

Credits:

here and here

If you need to recompile the modules because some of them is failing, you can do it this way:

vmware-modconfig --console --install-all

If you need more help and you want to get in touch with me Why not contact me ;-)

Errors compiling VMware Server on Fedora 12

Hi! Today i downloaded vmware server to use it on my Linux workstation.  I am using Fedora 12 with the latest kernel update so i had not much hope to compile the vmware kernel modules without some troubleshooting.

After solving the problems, here i share the solutions i hope to help you.  If you find it useful, remember that you can Invite me a kaffee ! :)

1. Download the latest VMware workstation (It is free)
https://www.vmware.com/tryvmware/p/activate.php?p=server20&lp=1&ext=1

Note: I downloaded the rpm version.

2. Install it:
yum install VMware-server-2.0.2-203138.i386.rpm

3. Try to compile the modules:
vmware-config.pl

Ok.. in this moment a lot of warnings and errors appear between them, errors with vmnet, vmci, etc...

make[1]: Leaving directory `/usr/src/kernels/2.6.32.10-90.fc12.i686'
Unable to build the vmnet module.


Unable to build the vmci module.

4. Patch the modules:

- Download from http://www.erdemap.com/store/2.6.32.10-90.fc12.vmmon.tar.gz
wget http://www.erdemap.com/store/2.6.32.10-90.fc12.vmmon.tar.gz

5. unzip & untar it.
6. execute the patch shell script *inspect it first, it is very VERY basic*
7. After installing the patches, execute the vmware-config.pl

Starting VMware services:
   Virtual machine monitor                                 [  OK  ]
   Virtual ethernet                                        [  OK  ]
   Bridged networking on /dev/vmnet0                       [  OK  ]
   Host-only networking on /dev/vmnet1 (background)        [  OK  ]
   DHCP server on /dev/vmnet1                              [  OK  ]
   Host-only networking on /dev/vmnet8 (background)        [  OK  ]
   DHCP server on /dev/vmnet8                              [  OK  ]
   NAT service on /dev/vmnet8                              [  OK  ]
   VMware Server Authentication Daemon (background)        [  OK  ]
   Shared Memory Available                                 [  OK  ]
Starting VMware management services:
   VMware Server Host Agent (background)                   [  OK  ]
   VMware Virtual Infrastructure Web Access
Starting VMware autostart virtual machines:

If you need more help and you want to get in touch with me
Why not contact me ;-)