# netstat -an | grep LISTEN | grep tcp
tcp        0      0 0.0.0.0:46631           0.0.0.0:*               LISTEN     
tcp        0      0 0.0.0.0:40072           0.0.0.0:*               LISTEN     
tcp        0      0 127.0.0.1:9001          0.0.0.0:*               LISTEN     
tcp        0      0 0.0.0.0:50060           0.0.0.0:*               LISTEN     
tcp        0      0 0.0.0.0:50030           0.0.0.0:*               LISTEN     
tcp        0      0 0.0.0.0:42004           0.0.0.0:*               LISTEN     
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN     
tcp6       0      0 :::22                   :::*                    LISTEN

There is nothing listening to port 9000 TCP. From the logs I see, tasktracker takes off without problem. But all the others are having problem.

Namenode couldn’t have started:

ERROR org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem initialization failed.
org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory /tmp/hadoop-localhost/dfs/name is in an inconsistent state: storage directory does not exist or is not accessible.
        at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:290)
        at org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:87)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:311)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:292)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:201)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:279)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:956)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:965)
INFO org.apache.hadoop.ipc.Server: Stopping server on 9000

Datanode says, it cannot conntect:

INFO org.apache.hadoop.ipc.RPC: Server at localhost/127.0.0.1:9000 not available yet, Zzzzz...
INFO org.apache.hadoop.ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 0 tim
e(s).
INFO org.apache.hadoop.ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 1 tim
e(s).
INFO org.apache.hadoop.ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 2 tim
e(s).
INFO org.apache.hadoop.ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 3 tim
e(s).

Of course secondary name node is off, too:

INFO org.apache.hadoop.ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 0 time(s).

I have looked up to Micheal Roll’s post about installing Hadoop on Ubuntu and tried once more with omitting IPv6 suppport. But no… Thereafter, have just looked up Hadoop Cluster Configuration guide. It points out options such as dfs.name.dir and dfs.data.dir. I think there is a problem with data files of Hadoop which are deleted since they are located under /tmp (locally) in default.

I have edit the conf/hdfs-site.xml to add these lines:

    <property>
         <name>dfs.name.dir</name>
         <value>/hometohadoop/hadoop-0.20.2/logs/transLogs</value>
    </property>
    <property>
         <name>dfs.data.dir</name>
         <value>/hometohadoop/hadoop-0.20.2/dataDir</value>
    </property>

Now I say Hadoop to store the relevant files under these directories instead of some unsteady directory under /tmp (local). Ensuring these folders do exist:

# mkdir -p /hometohadoop/hadoop-0.20.2/dataDir
# mkdir -p /hometohadoop/hadoop-0.20.2/logs/transLogs

I have restarted. It says now,

ERROR org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem initialization failed.
 java.io.IOException: NameNode is not formatted.

Which makes me format (delete all data, of course) in Hadoop so that I will eventually have a stable data store anymore:

# bin/hadoop namenode -format

Now restarting Hadoop again… And yet:

INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 9000: starting

***

Good…

***

I was looking for simulating a real (persistant) installation on the usb drive. And I came up with minimal installation.

1. Boot with either the Live CD, Minimal Installation CD or the DVD

I have rebooted with the minimal installation CD of Ubuntu Natty Narwhal 11.04 (If it’d be possible, better with its DVD).

I plug in the usb drive and re-detect the drives in the text based installation menu and do the configuration for the usb drive in the menu then install it.

2. Reinstall? :)

The trick is, when it is time to install GRUB it yields an error. So I restart and boot with the Ubuntu DVD open “rescue system” in the boot menu and go on with partition table (without any modification, but only configuring as in original, I have). It installs the system again.

3. GRUB install

I have booted from the DVD and used the line below for 11.04 (source: https://help.ubuntu.com/community/RecoveringUbuntuAfterInstallingWindows)

# sudo grub-install –boot-directory=/media/0d104aff-ec8c-44c8-b811-92b993823444/boot /dev/sda

After that,

# cd /media/???????

get into your usb drive and chroot to it:

# chroot .

then add a user within root group:

# useradd ?myname?

Just like that. Reboot and use the usb ubuntu. One more thing to remember: VMWare may not comply with a successful bootable USB Driver. I have wasted so much time before I understand that. I thought it was not complete just because VMWare did not boot from the USB which I have prepared. But no, though it works with real machine start up.
P.S.: If you need further help on VMWare have a look at this one: http://www.plop.at/en/bootmanagerdl.html

My portable LG HDD

I have noticed that portable applications are life saver, as traveling this much. I have reached resources such as portableapps.com and so forth.

However my problem was about the launcher itself. I have 2 GBs of portable applications on my traveling portable HDD. I have tried number of launchers, here are the notable ones and my sights:

1. portableapps.com Suite menu / portableapps.com Platform menu

The helper coming with the portableapps.com both bundles (suite or platform bundle) did well at first when I was discovering the world of portable applications. But I have noticed that they are insufficient for me. And free it is.

a. Startup Speed: Disaster, worst of all. It kills me to wait for like 1 minute to (USB 2.0) warm up. Because, application looks for new applications, icons, and stuff I guess. It is too much. I just want to load a Notepad++ maybe and waiting for 1-minute which is way too long if you consider this is “PORTABLE” not home necessarily.

In forums it is suggested to PAFize the application [2], which I don’t (neither want to learn and) want to do that for 2GBs existing already and the probable upcoming ones. Instead I need something with less installation cost.

b. Customizing Index Folders: Great, even the best. It automatically discovers the data inside my portableapps folder in HDD. Also supports dynamic drive letter of USB HDD which crucial because, I use a number of computers and each give different drive letter to my HDD. Such as, despite F:\portables is the folder at a friend’s, H:\portables at some other friend’s computer. It should go on working without any other operation inspite of computer.

c. Being Portable: Yes, thanks… Launcher should be portable of course, and it is.

d. Capability: Low. Just add applications and run them. That’s it. Serve for the purpose…

e. Instant Search Capability:  No. Just find the application on the list and click to open. Though it supports a “favourite applications” list above other all.

2. Executor

This is also a freeware for the same purpose.

Executor auto-complete example

a. Startup Speed: Good. Despite this is not really a hard to solve issue. But since it is customizable yes it reponds to that need well, it scanned and indexed in like 2 minutes and that is it. There is even an options to choose: index each startup, index each 15 mins, index each 15 mins of idle. Well done.

b. Customizing Index Folders: Yes, and good one.

c. Being Portable: Yes, you put $D$ in the drive letter to reach portable driver of yours.

d. Capability: High. Comes with few additional utilities built-in.

e. Instant Search Capability: Yes, and very good because it is customizable (really -highly).

f. Visual: Poor GUI (in sense of look only). I have seen better like Humanized Enso and Launchy.

3. Launchy

Open source and free.

a. Startup Speed: Good.

b. Customizing Index Folders: Yes, easy to manage, not bad. But when I get in to options and say “ok” each time it rescans whole directories, which is not that good. Sorry.

Launchy also has got great skins

c. Being Portable: Yes, kind of. Actually there is an impressing option in the installation asking “if this is to be portable or installed on system drive”. However no option to express portable driver is “portable drive” :) Sad thing is that they have that vision but ignore somehow to see that portable drives change letters mostly.

d. Capability: Medium. There are plugins you can even download more. But lack of main functions you expect in the first place.

e. Instant Search Capability: Yes, better than Executor. It has easily seen icons with dropdown.

f. Visual: Cool GUI (in sense of look only).

 But though, it has lack of support for dynamic drive letter. So no.

4. CodysSafe

When I realized that to add an application to the list is even harder than MS Windows itself, I quit trying. Overall, poor; for the first throat I should be passing.

5. SlickRun

Not to be so harsh, it does not fulfill my needs. But not to forget, this application is not an “portable applications launcher” it belongs to another category. But still, I wanted to try though. There is not support for customizing directory indexing.

6. Humanized Enso Launcher

It is free (anymore it says). Good one actually, if I wasn’t looking for portable solution. Again, this is a great software in fact. However it belongs to some other category as SlickRun. Because, it has great utilities builtin. But not to ignore, this is something you should use, if you know what you are doing. But though could watch the video here and others linked there. I would try to portabilize it :) if it supported customizing indexed directories. In portable meaning, it is poor despite great look and time saving GUI.

Conclusion

After all, I didn’t have time and courage to check FARR and maybe more stuff available on the net. However, I am going with Executor now.

[1] http://portableapps.com
[2] http://portableapps.com/node/25181
[3] http://executor.dk/
[4] http://www.launchy.net/index.php
[5] http://www.codyssey.com/
[6] http://www.fiddler2.com/SlickRun/
[7] http://humanized.com/enso
[8] http://humanized.com/enso/launcher/fingertips.php
[9] http://www.donationcoder.com/Software/Mouser/findrun/

So here is the first step: (just edit the PROCESS_NAME to the filename of yours)

watch -n 1 -p ‘ps -e -O “%cpu,cputime,sgi_p,%mem,rss,sz,vsz,start,nlwp” | grep -e PROCESS_NAME -e “%CPU” -e “###########” | sed “s/[ \t]\{1,\}/\t/g”‘

This was the tabbed version. Here is the version for human eye:

ps -e -O “%cpu,cputime,sgi_p,%mem,rss,sz,vsz,start,nlwp” | grep -e PROCESS_NAME -e “%CPU” -e “###########” | grep -v “###########”

Those # are just a trick not to grep grep itself.

I wanted to use gnuplot-py to sketch the realtime chart to observe CPU and MEM consumption. Nowadays, out of time. Maybe later, maybe someone finishes.

(Eclipse + ) Python + GTK+ + ClutterAfter

Eclipse

2 days non-stop working I couldn’t run even a simple Clutter project. I even couldn’t understand if Clutter will be under development further or not. There mentions on Clutter to be dead soon and deprecated under GObject.Bottomline, if I can’t even install it how the end user will… I give up with this option.

.

.

(Eclipse +) Python + PyDev + PySide  (+ Qt4) | (+ PyQt)

PySide is the Eclipse plugin to edit Python files under Eclipse. PySide and PyQt are two seperate projects for porting Qt into Python.

Qt

Qt has got amazing rendering tools within. I really thought “this is it” when I have seen Puzzle example [1] . Drag drop will be my main concern beneath other things.When I have seen PyQt is not available under LGPL (only GPLv1, GPLv2 and commercial which is not specified on the web site). I will be writing a Python code so it is nearly mandatory to share to codes, but, I don’t know somehow it suddenly looked “not-what-I-want” to me. So I checked PySide…

PySide is okay with carrying out LGPL outcomes.  But what confuses me is that PySide uses some of libqt libraries which are protected under GPL. Doesn’t it make PySide outcomes obligated to GPL? Ugh…

.

C + GTK+

I am really considering still to go with C because I don’t want to use C++ which looks like an unwanted child of C. I can’t say this is a thought I grasp after a deep technical analysis; I just didn’t like it. I would rather use Java if so. But C is not like that. It looks simpler to manage and more mature. It is not OO in nature but after recent modifications on C it is anymore usable as if OO supported. And speed attracts me indeed. And I don’t think porting to Windows or OSX would be problem as there are extremely great examples out there.

.

GPL / LGPL / Commercial?

GNU

I don’t have to share the codes. Of course I can share (I probably will) but I don’t want it to be mandatory for me — I don’t want to be obligated to share. Because, I don’t know how things will go on after first release. There is no income / donation plan in my mind for now. And obviously I need a business plan (even for non-profit option) to get this further which I don’t have one yet. I want to decide according to quality and need-satisfying of the project soon.

.

[1] http://gitorious.org/pyside-examples
[2]

Code highlighted in gedit:

Code itself:

#############################################################
### Protected and shared under GPL v2.0 license
### see: http://www.gnu.org/licenses/gpl-2.0.html
###
###
### Python code to gather structured information about
###   the work done as described on
###                       http://www.betterdesktop.org
###  which is great and promising about user experience
###                                    on free software
###
### edit: Dincer Kavraal
### mail: under "gmail.com" user "dkavraal"
#############################################################

from BeautifulSoup import BeautifulSoup
import urllib2

#------ init
print "Downloading...",
pageContent = urllib2.urlopen('http://www.betterdesktop.org/wiki/index.php?title=Data')
print "Done.\nRevealing...",
pageDom = BeautifulSoup(pageContent)
print "Done."
print "Content charcode:", pageDom.originalEncoding

#------ get all table:video-table
resultSet=[]
taskId=0
tables=pageDom.findAll("table", {"class": "video-table"})
for table in tables:
        pId=0
        resultSet.append([])
        resultSet[taskId].append(table.findPrevious('h3').string)
        for lineTag in table.findAll('tr'):
                arrayForPeopleCrted=False
                for columnTag in lineTag.findAll('td'):
                        if not(arrayForPeopleCrted):
                                pId+=1
                                resultSet[taskId].append([])
                                resultSet[taskId][pId] = []
                                arrayForPeopleCrted=True
                        if columnTag.find('a'):
                                resultSet[taskId][pId].append(columnTag.find('a').get('href'))
                        else:
                                resultSet[taskId][pId].append(columnTag.string.rstrip(' \n\r').lstrip(' \r\n'))
        taskId+=1
print "Total tasks processed: ", taskId
print

#------  print out summary table
taskId=0
for task in resultSet:
        taskId+=1
        print "---------------------------------------"
        print "".join(["[", str(taskId), "] ", task[0]])
        for pId in range(1, len(task)-1):
                print "".join(["\t\t", task[pId][3][0], task[pId][1], "(", task[pId][2], ")"])

#------ done.

Sample output:

---------------------------------------
[66] Task: Accept the offer to share an iFolder
		M18-30(9/10)
		M30-40(9/10)
		F18-30(5/10)
		M18-30(9/10)
		M18-30(9/10)
		M18-30(4/10)
		M18-30(9/10)
		M18-30(9/10)

i.e. Celtic music with a bio fits great or even maybe heavy metal beneath a fight scene. As if on a Hollywood movie…

Useless but promising :)

How do we decide what text is actually? Is it pessimistic or optimistic? Can a computer rate it for us?

For the beginning, can we rate a text is "good" or "bad" for the subject it is written for? How good? How bad?

Let’s see…

Work1 - Analysis

more to come…