Coder's view

Miyaesi - an automated music transcriber

2011-12-23T08:13:00.000-08:00

For last few months I was unable to write a blog post due to various reasons. May be because of my laziness on writing blog post. But I felt to write a post as my last post in this year, about our final year project which was completed couple of months back.

What is music transcription?

Music transcription is generating the music notation sheet out of a given input music recording. Following diagram shows the process of music transcription.

There are two basic types of music transcriptions.

Monophonic music transcription - In polyphonic music there can be only one instrument played at any given instance within the music recording. Monophonic music transcription is transcribing a monophonic music recording.
Polyphonic music transcription - In polyphonic music there can be several instruments played at any given instance in music recording. Polyphonic music transcription is transcribing a polyphonic music recording.

This automatic music transcription became and important topic in research field for many decades, due to the complexity of the methodologies and the technologies related to this domain. As final year students we were asked to handle this problem and try out a solution which address these complex issues. So we defined our objective for our final year project as research and develop a system which

Identifies notes
Identifies Instruments

and ultimately generating the notation sheet corresponding to a given monophonic music recording. "Miyaesi - an automatic music transcriber" became alive as a solution for above mentions issues, as a result of the immense effort taken by our team members for about a year.

Main features of Miyaesi are

Generating music notes out of music recording.

Detecting dominant instruments in each notes.

Creating a playable MIDI file.

Voice removal from music recordings.

Generating a printable notation sheet.

Estimation of a key signature.

Notes and MIDI file editing facilities.

Playing original and Midi files with notes displaying notes simultaneously.

Out of these features the most important feature is the instrument identification. Even though there are some research publications available on instrument classification area there are no good implementations for that. Miyaesi gain the advantage with this unique feature among other music transcribers.

Miyaesi is an opensource project where any one who is interested can contribute to the project. You can download Miyeasi and try its unique features.

https://sourceforge.net/projects/miyaesi/

AJAXify phpMyAdmin Interface - GSOC 2011 - 13th week

2011-08-21T10:34:00.000-07:00

This is the final week of gsoc 2011. As I mentioned in last weeks report I had to continue with rewriting the querywindow.js functions using jquey this week.

I had to rewrite the following functions this week.

PMA_queryAutoCommit()
PMA_querywindowResize()

Initially I rewrote the logic in "PMA_queryAutoCommit()" function using jquery, but I could not find a scenario where this function is triggered within the query window. Then I asked from Marc to get an ideal of a scenario where this function triggers. Then he suggested me that "clicking on the queries in sql history tab" should trigger this function. Further he mentioned that this scenario didn't work with his testings and if I also couldn't get it check correctly it is better to skip ajaxify this function. I tried the scenario and compiled the querywondow.js file with firebug, but it didn't trigger the "PMA_queryAutoCommit()" function. So I skipped ajaxifying the "PMA_queryAutoCommit()" fucntion.

Then I started working with ajaxify "PMA_querywindowResize()" function and with Marcs help I was able to test it correctly.

With that I finished ajaxifying the functions in querywindow.js file.

Then Marc directed me to a bug which is reported at bug tracker regarding the table->structure->change. When the user clicks on change a column which is having var_cahr type, the change dialog does not load the type correctly. So I went through the code and found this that this issue is not with my ajaxified part, but on original non table structure change section. Then I went though the tbl_properties.inc.php file and found what is the issue for this and reported it to Marc. Then Aris showed what was the bad commit which caused this issue and the remaining changes should do after discussing more with Marc.

With all these, I finished my tasks which were proposed my proposal at the beginning. Lots of people helped me a lot to achieve this success in the project, specially Marc, Ninad, Michal, Madura, Rouslan and Aris. Even after this gsoc project I'm willing to continue contributing to PMA which is a great pleasure to me.

AJAXify phpMyAdmin Interface - GSOC 2011 - 12th week

2011-08-15T00:34:00.000-07:00

According to the schedule I had to continue with ajaxify the table maintenance options in this week. Last week I did some modifications with this task and there were some bugs with that. Marc showed me that some important information are missing with the results of ajaxified table maintenance actions. In CHECK TABLE action there was a table which contains Msg_type and Msg_text information in non ajax configuration but in ajax configuration it was not there. In order to make this information available I had to do the following changes.

Added "if(isset($_REQUEST['ajax_request']) && isset($_REQUEST['table_mainrenance'])) {}" condition to sql.php file in order to handle the table maintenance action and return the results with those useful information.
Change the logic in "$("#tbl_maintenance.ajax li").live('click')" action to get the results sent by sql.php file and then show it in the table operations page using ajax.

Then after Marc showed me a new bug where the documentation links are not functioned properly, I changed the "$("#tbl_maintenance.ajax li").live('click')" action to "$("#tbl_maintenance.ajax li a.maintain_action").live('click')" and changed the tbl_operations.js file according to that.

After all these changes this option seems working properly and I'm waiting for the response by Marc.

Since I skip ajaxify the "Table options" in table operations after discussing with Marc due to inefficiency of doing that, Marc asked me to find a new task instead of that. I went trough various options in PMA and could not find a suitable action which is really need ajax behavior at this moment. Then I went through the "Ajaxify Interface" section in PMA wiki and found that the ajaxification of table insert is remaining as a to do task. After discussing with Marc he mentioned that this is not necessary at this stage and he asked me to involve with more useful task which is rewrite the querywindow.js file using jquery.

The querywindow.js file contains four function which are

PMA_queryAutoCommit()
PMA_querywindowCommit(tab)
PMA_querywindowSetFocus()
PMA_querywindowResize().

I rewrote the 2nd and 3rd functions using jquery and works fine for testing.

Next week I'll continue with rewriting the querywindow.js functions.

AJAXify phpMyAdmin Interface - GSOC 2011 - 11th week

2011-08-07T22:04:00.001-07:00

According to the schedule I had to continue with ajaxifying table operations task this week.

Since there was a bug with "Table copy" option which was done last week, Marc asked me to fixed the issue which is currently not reloading the navigation panel after copping the table to a new database. Initially I could not understand why we need a reload on navigation panel, but after the Madhura clarify me why we need it I fixed the issue.

Then I start working with ajaxifyig the other four options in "Table operations". Those are

Move table to database
Table options
Table maintenance
Delete data or table.

After discuss with Marc we decided "Move table to database" option do not need ajax behavior, because anyhow we have to redirect to the moved table after moving the table to new database. Also we decided that only "Empty the table(Truncate)" option in the "Delete data or table" section need ajaxification because after delete action there is no need to be in the Table operations of the deleted table.

I started ajaxifing the four actions in Table maintenance section.

I added "$("#tbl_maintenance.ajax li").live('click')" ajax action to the fucntions.js file which handles all four ajax click actions in "Table maintenance" section, and get the results of those actions.
Then I changed the tbl_operations.php and sql.php files in order to handle the ajax request and response according to that.

Then Marc showed a me a bug of not showing the Msg_type and Msg_txt feature after "Check table option" and I'll fix it next week.

Then I worked with Ajaxify the "Empty the table (Truncate)" option.

Since there is some related code for this feature with $("#truncate_tbl_anchor.ajax").live('click')" action in functions.js file I reused that action and changed the logic to show the results using ajax.

Next week I 'll continue with ajaxify the "Table options" and fixing the bug in "Table maintenance".

AJAXify phpMyAdmin Interface - GSOC 2011 - 10th week

2011-07-31T17:55:00.000-07:00

According to my timeline, this week I had to work with ajaxify the options in table operations. Mainly this week I focused on ajaxify two option in the table options. Those options are

Alter order by
Copy table.

For the "Alter order by" option I did following changes to handle the ajax request and the response.

I added the "$("#alterTableOrderby.ajax").live('submit')" action to the functions.js file which posts the alterTableOrderby form with ajax post and then return the results without reloading the page.
I modified the tbl_operations.php and tbl_alter.php files in order to handle the ajax request which was sent by the above action.

Now when the user click on the go button "Alter order by" form, it get submit as a ajax request and then show the results of the action to the user without reloading the page.

Then I started working with ajaxify the "Copy table" option in the table operations. I did following changes in order to handle this ajax response.

I added "$("#copyTable.ajax").live('submit',)" action to the functions.js file which handles the ajax submission of the copy table form. When the "Switch to copied table" options is selected I continued with the non ajax submit.
Then I modified the tbl_operations.js file and the tbl_move_copy.php file which handles that ajax request.

After doing these modifications now when user copy a table to another database he can do it by reloading the page and if only the user select "Switch to copied table" option only he redirects to the copied table.

Next week I'll continue with ajaxify the remaining options in table operations.

AJAXify phpMyAdmin Interface - GSOC 2011 - ninth week

2011-07-24T10:09:00.000-07:00

This week I continued my works on "Ajaxify the Add column(s) in table structure". I added two ajax actions in to tbl_strucure.js and functions.js files.

I added the "$("#addColumns.ajax input[value=Go]").live('click')" action to the tbl_structure.js file which gives the add columns dialog for the user.

Then I reused the code in "$("#append_fields_form input[name=do_save_data]").live('click')" action in functions.js file which was initially written for handling the table->change option. I did small changes for the same action and used with this feature.

Then after Marc pointed out two issues with the enum editor which are

Cannot edit the enum editor values
Not appearing the name of the column in enum editor header

Since I have fixed a similar issue in Table->Structure->change feature I used the same logic of overlaying the enum editor by changing its z index using jquery.

Finally when the feature is finished, user gets an "Add column" dialog box which allows the user to add a column to the table as follows.

I started working with modifying the Table operations tasks by applying the ajax behavior for those features. I started ajaxifying the Alter table order option and still in progress.

Next week I'll continue with that feature.

AJAXify phpMyAdmin Interface - GSOC 2011 - eighth week

2011-07-17T23:10:00.000-07:00

The mind term evaluation for the GOSC 2011 with phpMyadmin was on this week and I was able to successfully get through the evaluation this time with the help of Marc and the other community members.
In this week I worked with modifying the initTooltips() function which converts the footnotes to tool tips. I changed the logic by defining the relevant domain of its operation at each time by adding and extra parameter to it. But Marc informed me on some issues with that logic and I'll work on that in next week.

Then I worked with my 9th week task "Ajaxify the Add column(s) in Table structure". I added the "Add column" dialog which carries the "append_fields_form" for submitting action in the add column form in the table structure.

There are some modifications with this option and I'll push my code as soon as I finish them.

Next week I'll continue with above work and start working with "Implement the Table operations tasks"

AJAXify phpMyAdmin Interface - GSOC 2011 - Seventh week

2011-07-10T17:49:00.000-07:00

This week my target was to work with "ajaxify the Column->Change in Table->Structure" task. But at the beginning of the week Rouslan and Marc arose two issues regarding my previous modifications. Those are

Set of issues with the "Ajaxified Index->Edit" option
An issue of not appearing the date time pickers in "DB->Structure->Insert"

With the issues in "Ajaxified Index->Edit" I had to rewrite the "checkIndexName()" function which was written in the indexes.js file. The existed method was written in pure javascript and I had to write it using jquery. And there were some issues with the manner I have used the jquery dialog and by Rouslans explanation I was able to gain some important facts on using the dialog correctly.

Then I worked with DB->Structure->Insert" issue with the help of Rouslans help and was able to bind the datetime picker with the insert form.

Then I start working with my this weeks task which is to "ajaxify the Column->Change in Table->Structure". Initially I tried to reuse the code which I have used in my previous task "Ajaxify Table->Structure->Muli_column->Change". So I moved the common code to "changeColumns(action,url)" function and used with both

$("#fieldsForm.ajax .mult_submit[value=change]").live('click')
$("#fieldsForm.ajax #tablestructure tbody tr td.edit a").live('click')

tasks.

Since I used "modal:true" parameter within the dialog there was an issue of "enum-editor" getting disable. This happens because of applying the "ui-widget-overlay" for the rest while we editing in the dialog. I found that the basic z-index of the overlay id 1000 and dynamically changed the z-index of the "enum-editor" to 1100 so it appears above the overlay.

Now it works properly and the modifications were merged to the master branch by Marc.

Change dialog box

Enum editor

Next week I will work with the mid-term evaluation and implementing "Add->Columns" in "Table->Structure".

AJAXify phpMyAdmin Interface - GSOC 2011 - Sixth week

2011-07-03T22:09:00.000-07:00

This week I mainly worked with three options in my project. Those are ajaxifying the following features.

Continued with "Multi_row ->Export" in "Table->Browse"
"Indexes->Edit" in "Table->Structure"
"Hide/Show Indexes" in "Table->Structure"

The first task is a continuation of the my forth week task. In this I tried to export the results to an external file and tried to do it with ajax. But after referring some discussion in forums I realized that this is hard to achieve with ajax and then after discussing with Marc I have reverted the changes done to the ''Export" option and skipped the "Multi_row -> Export" in "Table->Browse" option to been ajaxifying. But I suggested a Jquery plugin for future developers, which is able to solve this issue.

Then I started working with my second task which is adding ajax dialog box for "Indexes->Edit" feature in "Table->Structure". I mainly used "tbl_structure.js" file to handle the logic of actions and modified the "tbl_indexes.php" file to response for ajax request for this. In tbl_strucuture.js file i used three actions to handle this feature. Those are

$("#table_index tbody tr td.edit_index.ajax").live('click') - pops up the "Edit" dialog which loads the "index_frm"
$("#index_frm.ajax input[name=do_save_data]").live('click') - submits the index changes for the "tbl_indexes.php" file
$("#index_frm.ajax input[name=add_fields").live('click') - Add more columns to the index form

After completing above modifications this feature works fine and add more usability to the index edit option.

Edit dialogbox

Then I started working with my third task which is adding "Hide/Show Indexes" in "Table->Structure". I used tbl_structure.js file to handle the actions logic for this. I have generated "#toggletableindexdiv" which carries the "toggletableindexlink" which will toggle the index table show and hide. Following screen shots show how this feature works.

When the indexes are shown

When the indexes shown

After adding these features to PMA, it improved the usability and allow the user to do above mentioned actions without reloading its base pages.

Next week I'll work with ajaxifying the "Single_column -> Change" in "Table->Structure".

AJAXify phpMyAdmin Interface - GSOC 2011 - Fifth week

2011-06-26T09:09:00.000-07:00

Since I could not finish my fourth weeks second task which is "Ajaxify the multi row export in table browse" last week, I continued working with it in this week.

I used the sql.js file for handling the actions related to export and then the sql.php file. For this task I tried to reuse the toggle_quick_or_custom(); function. I used two actions for this task which are

$("#resultsForm.ajax .mult_submit[value=export]").live('click') which pop ups the "Export" dialog box containing the "dump" form for export
$("#buttonGo.ajax").live('click') which post the export data to export.php file.

Even though I worked with this task still there are some bugs with that option and I'll continue with that in the next week also.

Even tough my "Ajaxify Table-> Browse->Multiple rows->Export" option is not yet finished, since my this weeks task is "Ajaxify Table->Structure->Multiple rows->Change", I worked with it.

In this I tried to reuse the "if($("input[type='hidden'][name='table_type']").val() == "table")" code segment and place it in "displayMoreTableOpts()" fucntion in function.js.

I used the tbl_structure.js for handling the change event and in that I used two actions which are

"$("#fieldsForm.ajax .mult_submit[value=change]").live('click')" action to pop up the change dialog which contains the "#apend_field_form".
"$("#append_fields_form input[name=do_save_data].ajax").live('click')" action to handle the save option for save the changes.

There small modifications which should done for this option, so I need to continue with this option in the next week.

My sixth week tasks are "Ajaxify edit index and show/hide index in table structure", so I'll work with those options in next week.

AJAXify phpMyAdmin Interface - GSOC 2011 - Fourth week

2011-06-20T10:42:00.000-07:00

After ajaxifying the table-> insert in db-> structure in last week, I had to fix some bugs in the same option this week also. Then I started applying ajax dialog for muti_row->change in the table->browse option. As Marc requested I have changed the name of the form "rowsDeleteForm" in to "resultsForm". Then I went through the code to find whether there are any code segments which can be used for my developments. There were not much code on that and I just find the

$("input[name=navig].ajax").click

function which can be used for reload the "resultForm" after successfully changing the row values.

Then I started implementing the feature and first I developed the click event for "Change" option which pops up the "Change" dialog which contains the "#insertForm" form which allows to change the selected values.

$("#resultsForm.ajax .mult_submit[value=edit]").live('click')

Then I developed the two events which handles the clicks on go buttons at "InsertRowTable" and "navigationPanel". Those are

$("#insertForm .insertRowTable.ajax input[value=Go]").live('click') and

$("#buttonYes.ajax").live('click')

On those I used the "$.post()" action to submit the changed values and if it is success used to trigger the "navg" button to reload the "resutlsForm".

$("input[name=navig].ajax").trigger('click')

After instructions given by Marc, I changed the code in order it supports for the vertical table.

This feature is now works fine and merged to the master branch by Marc and I believe it really improve the usability of the PMA.

Next week I have to work with ajaxify the table->export in db->browse and the actions in multiple rows in the table->structure.

AJAXify phpMyAdmin Interface - GSOC 2011 - Third week

2011-06-13T08:40:00.000-07:00

After finishing my last week task which is ajaxifying the "Add a new User" in db->privileges, this week I started providing an ajax featured dialog to the table inserts in db->structure.
There were no previously done related works on that, so I had to start it form the beginning and there were no any codes which I could reuse in my implementation. I chose the db_structure.js file to handle the ajax behavior in this insertion.

Initially I added the ajax class for the insert link and then wrote the action for that insert link click in db_structure.js file. The action is $("td.insert_table a.ajax").live('click'). After click action it popups the "insert_table_dialog" dialog which carries the "insertForm"

Insert Table pop up dialog

Then I implemented the action which triggers when the "Go" button in "insertRowTable" is clicked. That is $("#insertForm .insertRowTable.ajax input[value=Go]").live('click') and it basically post the insert form to the db_structure.php and then after the record is successfully inserted it reloads the "tablesForm" in the db->structure with newly inserted values. I used "$().load()" function to load the tableForm with new values.
Then I implemented the click action of the "Go" button at the "action_panel" in the insertForm. That is "$("#buttonYes.ajax").live('click')". In that case I had to handle the different "submit_types" and "after_insert" types. When user select the "new_insert" in after_insert list the "Insert table" dialog should re popup. So I used "trigger('click')" function to trigger the insert click automatically.

Now the insert option in db->structure works fine with its ajax behavior but seems little bit slow due to high number of "get()" requests during the operations. I'll fix that issue in the next week.

Next week my major tasks will be implementing the Actions on multiple rows in table browse. It will basically contains two part which are "change" and "export".

AJAXify phpMyAdmin Interface - GSOC 2011 - Second week

2011-06-05T09:30:00.000-07:00

After successfully completing my first task which is Ajaxify the "Create Table" option in the navigation panel last week, I moved to the second task of the schedule. It is applying ajax behavior to the "Add a new User" option in the db->privileges tab. Initially I went through the implementation of "Add a new User" option in host->privileges which is a similar situation as db->privileges. I went through the code in the server_privileges.php and server_privileges.js files to find out what are the available logics and the code which can be reused for my implementation. I found that $("#fieldset_add_user a.ajax").live("click") event code is the logic and the code which I can reuse for my implementation. After that I started modifying it according to my task.

First I added the class="ajax" attribute to the "Add a new user link" when the Ajax in enabled. Then It became supporting to the same logic in $("#fieldset_add_user a.ajax").live("click") event.

At that point when we click on the "Add a new User" link, it successfully popups the dialog box which is carrying the "Add user" form and after we submit it correctly create that user. But still its not done. Because I had to reload the "db specific user rights" table at the db->privileges tab.

After discussing with Marc I decided to use a post command to get the updated "db specific user rights" table in db->privileges and replace it with the current table. So in oder to do so I had to remove the echoes of the "db specific user rights" form in server_privileges.php file and use the PMA_ajaxResponse() to pass the string of the table object as a json string.

Then I had to face a problem of not parsing the json string correctly and outputs error. So I used the $.parseJSON() function to parse the json string at the server_privileges.js file. Then it worked fine and reloaded the "db specific user rights" table with newly added user.

Then Marc directed me to a new scenario in host->databases->check_privileges->"Add a new user" which gave some errors because of my modification. Finally I have modified the logic which corrects the error and now the feature is working correctly.

Add user dialog

Now the "Add a new user" in db->privileges option works fine with the ajax behavior and I think it will help to improve the usability of PMA.

Next week I'm going to work with modifying the "Insert for a table" option in db->structure, which will apply the ajax behavior to it.

AJAXify phpMyAdmin Interface - GSOC 2011 - First week

2011-05-27T10:17:00.000-07:00

This is the fist week of coding time period in my gsoc 2011 project with phpMyAdmin(PMA). On 23rd of May the official coding time started. Since there were some free time with my in the community building period I started my works on before hand. It made me easy to be familiar with the code base and style which is used in PMA.

My first task is give the ajax behavior to the "Create Table" option in the navigation panel of the database. Initially when we click on the "Create Table" option it directs the content frame to a new page. So I wanted to give a dialog box which loads the create table form and then submit it using ajax.

First I went through a similar feature in the "database->structure" and "database->operation" by using the "create_table_form_minimal" form. At that case I got some knowledge on how we can use dialog boxes for this kind of scenario. Then I tried to find out of them what are the code segments which I can reuse for my task. There were three actions which used for "Create Table" option in "function.js". Those are

("#create_table_form_minimal").live('submit') - triggers when the initial form has submit.
("#create_table_form input[name=do_save_data]").live('click') - triggers when the save button clicks in the form.
("#create_table_form.ajax input[name=submit_num_fields]").live('click') - triggers when the add new column option clicks.

Out of those I realized that the second and third actions are common to both cases. I had to add a new action instead of the first one.

Then I planed how to structure my new option. After discussing with Marc I decided to add my code the "navigation.js" so it is logical and easy to use in the future. Then I developed the (#newtable).live('click') action in "naviation.js" file and after discuss with Marc I put the common logic to the "PMA_createTableDialog()" function in the "fucntion.js" file.
Finally Marc requested a modification on changing the "Add 'x' column(s) Go" option to the top of the create table form and I did that. Now the "Create Table" option in the navigation panel works fine and I think the modification added some more usability on that.
I have successfully finished my first task of my project. Next I 'm willing to work with adding the ajax behavior to the "database->privileges" option.

AJAXify phpMyAdmin Interface - Gsoc 2011

2011-04-30T11:15:00.000-07:00

This time my proposal on "AJAXify phpMyAdmin Interface" got accepted for Google Summer of Competition with phpMyAdmin (PMA). Marc Delisle, Michal Čihař and Ninad Pundalik who is the last year Gsoc contributor helped me a lot when I improve my patches and the proposal. Marc Delisle act as my mentor for my project in this summer.

Ajaxifing the PMA is a project idea which continues from the last year gsoc session which was done by Ninad Pundalik. Last year he has improved some important features by giving the ajax behaviour to them in the PMA. Since ajax is a trendy behaviour among web domain and since it improves the usability of the system, moving in to ajax is a huge improvement step on PMA. This time I’m willing to improve the usability of some identified important features in PMA by applying ajax behavior using jquery library. After improving those features, they will be able to provide a quick and consistent flow of operations to the PMA users.

Next series of blog post will discuss more about my project in this summer.

Optical Character recognition with Tesseract

2011-04-23T09:30:00.000-07:00

Since I was very busy with my final year stuff I could not write a blog post for months. But today I got some time to introduce an amazing tool for you, which will help you when dealing with the Optical Character Recognition(OCR) domain. Lets get a basic knowledge on the OCR domain, before moving to Tesseract.

What is OCR?

"Translating the handwritten, typewritten or printed texts in a scanned image to a machine encoded text."

This OCR concept is very important and widely used in converting important documents which are in image formats to text format. The first major use of OCR was in processing petroleum credit card sales drafts. This application provides recognition of the purchaser from
the imprinted credit card account number and the introduction of a transaction. Most of the modern scanners provides the feature of OCR enabling for the images scanned using it which will directly output the text content of the scanned image. This concept helps to save a huge time which will waste on rewriting the image contents. There are lots of pattern recognition and image processing activities involved with this field and this is an very interesting and research area in the current computing word.

What is Tesseract?

"The Tesseract OCR engine was one of the top 3 engines in the 1995 UNLV Accuracy test and it is probably one of the most accurate open source OCR engines available. The source code will read a binary, grey or color image and output text. A tiff reader is built in that will read uncompressed TIFF images. Tesseract is licensed under the apace license and any one can use it for free. The core developer on the project is Ray Smith "

Sometimes back I involved with a project to modifying an OCR module for Sahana Disaster Management System. So I searched for an OCR engine for integrate it with the SahanaOCR system. Tesseract was the best solution I found. It has an immense power on recognizing printed text and also it is very easy to localize for your own mother languages.

How to install Tesseract on your PC ?.

Tessearct provides a set of binary executable for Windows and installers for Linux systems. The detailed installation guild is available here.

The Windows users can directly download the latest exe file here. The installation guide for the Linux users also provided at the general installation guide . Then you have to download the trained data folder which is available at the download page. Currently there are trained data sets for many languages such as English, Chines, Spanish etc. You can download any of those tessdata files and extract it into your working directory.

Then there you can go into the directory where the tesseract.exe is located and use the following command to recognize the images and finally to output the recognized characters in an text file.

tesseract image outputbasename [-l lang] [configs]

In this the image file is name of your image using for OCR and the output_base_name is the name of the text file which will contains the recognized output. The images should be in TIFF format and you can use the Libtiff library for handling compressed images with Tesseract. Other configurations are optional and they facilitate for more advance features.

Training Tesseract.

There is another amazing feature provided by Tesseract which is customizing it for your local languages. For that initially we have to train Tessearct for the new language. Here is the training guide for the Tessearct.

For training it, you have to have a proper training set of characters. Then after you select a proper training set you can follow the training procedure which is described at the guide. After following all the steps it will provide you a lang.trainddata file which contains all the newly trained data.

Tessbase API.

Now you have a good idea on how to use Tesseract for OCR. But thats all for using Tesseract on its own. What if we need to integrate it with our own application. So for that Tesseract provides a highly user friendly API which is named as Tessbase API. So you can use tessbaseAPI class to call the internal functions provided by Tesseract.

Here is the general forum where the common discussions carried out on Tesseract project and you can find more details on the project by subscribing to that.

My project on SahanaOCR Gsoc 2010

2010-10-27T09:30:00.001-07:00

This is my first post after about 8 months from my last post. Since I was very busy on the training period and the Gsoc competition I was unable to write a post o my blog. How ever again I managed to find some time to write a blogpost fortunately. So I decided to continue the blogpost on SahanaOCR project from where I have stopped at my last post.

So as I mentioned earlier since I have worked with the SahanaOCR system before I came up with the idea of applying for the Gsoc competition for the SahanaOCR project. So I have wrote a proposal on the project with discussing the major implementations and developments that I'm going to do in the project in the session. Here is my project proposal, So you may have a better idea on how to write acceptable project proposal for the Gsoc competition.

My project extension plan was to improve the most necessary features of the current project. The proposal propose 5 main ideas for the existinng Sahana OCR project

Make the system platform independent
Integrate the Tesseract code to the project
Adding the feature of differentiating the different forms by a bar-code embedded to the forms automatically handles those forms.
Read the labels of the data fields and identify the type of data field so can give restrictions to the Tesseract to identify the numbers or letters separately.
Automatically send the data to the databases of the relevant modules

So after completing these steps the Sahana OCR project will be able to complete the process of extracting data from forms with a great accuracy.

After being active on the Sahana mailing list and exchanged lot of ideas on the project with earlier mentors and contributors they have accepted my project proposal for the Gsoc 2010 competition. that was a day which gave me the same happiness as I passed my Advance Level exam. My mentor was Mr. Gihan Chamara who was graduated from my department last year. During the whole project he was the great strength for me and also Mr chamindra, Mr Jo and Mr Hayesha guided me to make this task success. I have started my project implementations on the 24^th May and it gave me lot of memorable experiences to my life during the whole project time which ran up to 10^th September.

It taught me how to plane a project, how to overcome the challenges we met during that, how to behave withing the public communities, how to get helps form others as well as some more things which were new to me.

Here is the wiki page that I have maintained during the project and you can get the more information on the technical details and how the project ran by referring that.

Here is the link for my project report which I have submitted at the end of the project period and it contains all the details of the current status of the project.

Here is my project branch which I have submitted my code in the Launchpad.

Even though the Gsoc competition and over my effort on the SahanaOCR has not over. So my wish is to continue on working with it and be with it until it became a applicable standalone product.

Sahana Ocr

2010-02-28T05:28:00.000-08:00

I have explained about the Sahana project I my previous blog post. So today I'm going to explain about the Sahana OCR project which is an sub module in the main Sahana project. You can refer my research paper about the extension I did for this project.

Sahana Ocr project was done to automate the data entering process to the system and it was developed on .net framework using Visual C++. It was developed by many volunteers as well as gsoc contributers as Omega and Gihan in past two years. I have selected this project as my level 3 project and did some improvements to the existed system to improve the character recognition of the module.

After a disaster situation we have to collect the data about the victims and manage them using Sahana as I mentioned in my earlier post. So we can download the form template by the Sahana server, print the forms in to hard copies, distribute the forms to the victims, ask them to fill the forms and collect the forms back. Then we have to enter those data manually to the Sahana system. But in the case of a large disaster the manually data entering process is difficult and very tome consuming. So Sahana Ocr project is developed to automate that data entering process by using optical character recognition (OCR) technology.

This is a sample of the forms distributed to the victims and you can refer the wiki abou this module to get a better understanding about the forms and the xforms. sahana_xform

So in this we have to provide the scanned image of the filled form and a xml file called xform which have the coordinates of the data fields in the template if the form to the Sahana Ocr module. Then the module sent the two files to the Form_processor which is the manager class of the system. First Form_processor do the rotation compensation to the image. The system uses five back squares at the form to identify the right side of the form. Then it validate the form image with the xform whether the correct xform coordinates are in the form image. If it is not the system gives an error.

After validating the image the image was sent to the the image processor and the image processor finds the Input areas of the form then data fields within them and finally letter boxes according to the xforms coordinates. The data fields, input areas and letter boxes are explained in the above image.

The image processor was developed using the OpenCv libraries and it worked greatly in this project. You can find more abou the OpenCv libraries by referring here.

Then those letter boxes are sent to the Ocr class which is capable to recognize the character in that letter box. The initial Ocr class was developed using Fann neural network libraries and the accuracy of the recognitions was very bad because of the lack of training of the neural network. That’s the point that I contributed to this project. After discussing with the previous mentors and the developers of this project I have used Tesseract Ocr engine which was an Google product to replace the Fann neural network.

You can get information about the Tesserat from here.

I did some testing with the Tesseract for the hand written characters and check whether it is suitable for our module. The initial accuracy of the recognition with the Fann Library was below 30% and when it was tested with the Tesseract it was increased up to more than 80% and the worst case with was 80% when I tested for different data sets. I realized that it is a very powerful tool and selected it for the project. Then I have customized the Tesseract to make compatible with the expectations and changed the Form-processor in order to make it compatible with the Tesseract.

Finally the recognized letters are consolidated as strings and assign them to the appropriate labels such as Name, Address and Date of Birth. Then I have developed a GUI for the module which gives a graphical view for this process. Then the results are display at the GUIs result tab window. Then it allows the users to save the data in a file by submitting it.

So it was the greatest project that I was involved up to now because it gave me a lots of knowledge and it was an great opportunity made contacts with some interesting personalities involved in that project. I have an idea to work on this project more and extend this project to make the accuracy at least up to 97% and make the rest of the project as uploading the data in to the server directly kind of things. These some more ideas with me to work further on this project.

· Train the Tesseract more for hand written characters

· Increasing the accuracy by reading the labels. That means currently the OCR only reads the letters at the letter boxes and output the result. But in that case there are conflicts when recognizing the “0” and ”O” which the first one is zero and the second one is letter O. So when we read the label we can give a logic that the letter boxes under the Name label cannot have the zero number and also the letter boxes under the date of birth cannot have the letter O with it. So in that way using label reading we can further improve the accuracy.

· Making the data embedded with the Url given to the each form. That may make easy to transfer data from one place to another.

So please try on this project and try to make some innovative ideas to serve the people in the world in a situation that they are in need.

You can refer some more articles about this project from here.

How link another folder to your www web folder.

2010-02-24T20:30:00.000-08:00

In last post I have describe the way to link your public_html folder to the www folder. To I will describe a simple way to link a folder to your /var/www to make easy your web developing stuff on linux.
1.To do that first you have to make a folder in your favorite place using mkdir command.
mkdir /home/abc/myFolder

2.Then go to your /var/www/ folder.
cd /var/www

3.Then link your folder to the www folder using ln command.
sudo ln -s /home/abc/myFolder

This will make an linking file in the www folder and you can see it in your www folder in the same name as you used for your linked folder

Now you can use your myFolder folder for run files in the web server by the address
/http://localhost/myFolder/

So use this method to make easy of your life in using servers for advance developments.

What is public HTML

2010-02-24T20:21:00.000-08:00

If you are working with any kind of web server on linux operating system you might have suffered enough from the warning massages saying “This is write disabled file, please make the file write enable”, “Cannot move the file in to www folder” so on. So is it a headache for you. Here is an great and simple solution for your problem.

In linux most of the situations you have to copy your files such as html and php, in to the /var/www/ folder to run them on your favorite web servers such as apache web server. But this folder is protected and you need some permission to copy, remove or change files in the www folder. So when we are involve in a web development using large number of files moving here and there it is difficult to work in this kind of environment.

So now you can use another folder that is away from www folder which is not a protected one as your www folder. It will help you to make changes in the folder as you wish and you can get the same functions as in the /var/www folder. Following are the steps of making a public_html folder and make it enable for access by your user-name as http://localhost/~ABC.

make a folder in your favorite place in your folder structure. But make sure it is not a protected folder. As an example make a folder in the user folder by using the following command in command prompt:

sudo mkdir /home/ABC/public_html/

Go to the folder of your web server. As an example if you are using apache2 web server you can use following command

cd /etc/apache2/

Then link your public_html folder to the web server.

sudo cp -r mods-available/userdir.* mods-enabled/

Then restart your web server

sudo /etc/init.d/apache2 restart

Now you are ready to use your public_html folder as your web servers working folder.

You can access the folder by typing the following address on the address bar.

http://localhost/~ABC

The most important thing is if you are using your computer connected to a network and if others are access to your computer via network you can use this folder as the access folder to the out side, so the others can access to that folder with out permission enabled as well as you can keep your protected data away from this folder so others can't access those. Other wise if you enabled permission in the www folder the others may access to some other important data in your computer. The others can access to this folder by the address of your Internet address. As an example

http://www.your_net_address.com/~ABC

So please try this and feel the difference.

Sahana foss disaster management system

2010-02-03T00:51:00.000-08:00

Sahana is an open source disaster management system which is deployed to manage number of disaster situations all over the world.

Initially it was developed to manage the situation occurred in the Indian Ocean tsunami disaster which was occurred in the 26^th of December in 2009. Lanka software foundations did the initial development of the Sahana phase-1 with the help of many organizations and the volunteers from various parts of the world. At that time there were no other free software which can support to handle all the situations as handling victims, managing camps, manging the volunteers, manging the NGOs like wise. So this was a better solution for all of this issues.

Initially Sahana contained 7 modules to help the management of the disaster. Those are

Missing Person Registry - Helping to reduce trauma by effectively finding missing persons
Organization Registry - Coordinating and balancing the distribution of relief organizations in the affected areas and connecting relief groups allowing them to operate as one
Request Management System - Registering and Tracking all incoming requests for support and relief up to fulfillment and helping donors connect to relief requirements
Camp Registry - Tracking the location and numbers of victims in the various camps and temporary shelters setup all around the affected area
Volunteer Management - Coordinate the contact info, skills, assignments and availability of volunteers and responders
Inventory Management - Tracking the location, quantities, expiry of supplies stored for utilization in a disaster
Situation Awareness - Providing a GIS overview of the situation at hand for the benefit of the decision makers

But now it has more than 20 modules and day by day it is increasing its scope to be more reliable and useful for the society.

You can visit here and get an overview about the project.

This was successfully deployed in the following situations occurred in the past years .

Tsunami - Sri Lanka 2005 - Officially deployed in the CNO for the Government of Sri Lanka
AsianQuake - Pakistan 2005 - Officially deployed within with NADRA for the Government of Pakistan
Southern Leyte Mudslide Disaster - Philippines 2006 - Officially deployed with the NDCC and ODC for the Government of Philippines
Sarvodaya - Sri Lanka 2006 - Deployed for Sri Lanka's largest NGO
Terre des Hommes - Sri Lanka 2006 - Deployed with new Child Protection Module
Yogjarkata Earthquake - Indonesia 2006 - Deployed by ACS, urRemote and Indonesian whitewater association and Indonesian Rescue Source
New York City - 2007-08 – Pre-deployed in support of the City of New York’s Coastal Storm Plan
Peru Ica Earthquake - 2007 - Deployed for the Government of Peru
Chendu - Shizuan Province Earthquake 2008 - Deployed by Chendgu Police
Haiti earth quack 2010

This is an web base system and the initial developments were done in Apache, MySQL, and PHP/Perl. But now it is redeveloped using python thechnology and it was callled as SahanaPy.

Now the Sahana software Foundation was initiated to take care of the Sahana project and it will be a great step for the future of Sahana.

The source code can be downloaded from here using CVS

cvs -z3 -d:pserver:anonymous@sahana.cvs.sourceforge.net:/cvsroot/sahana co -r gsoc_2009 -P sahana

and the it can be run using the wamp/lamp server. Here are some tips to run the source code.

When it initially set up, it automatically creates a data base and the accordance tables in the server so now it is ready to give it's services.

So try it and join to the Sahana community to help the people when the really needs you.

2009-12-10T19:08:00.001-08:00

what is DIGSBY

2009-06-09T21:03:00.000-07:00

Since I have couple of accounts in various community sites and in the mail servers I had a problem of sing in to all those accounts and check whether there are any mails have dropped for me or there are any updates in my profile. But now I found a solution to that which helps me to get all the notifications of the mails and the accounts. It is the DIGSBY software which can be download it from here.

You can enter your account passwords in to this software and when you connect to the internet it automatically give you the updates of your accounts. So try it. It will make your works easy.

How to add a Statcounter to your blog

2009-05-28T08:36:00.000-07:00

Hi friends. Do you know how to insert a gadget to your blog to keep track of the visitors who visited to your blog. There are lots of web sites which are offers this facility freely.

statcounter.com is a good web site for that. You can make your own account in that and after that you can choose the gadget which is suitable for your blog. Then add a html/JSP gadget in your blog and past the code provided by the website on that. It will make nice tool on your blog which shows you the number of visitors. Try it………

There is another website which will give you a visitor counter freely. It is FlagCounter. Try it and get the new gadget ….

My mini project on search engine

2009-05-16T21:57:00.000-07:00

I’m at my level two-semester two of my university life. Last month I was very busy after my new year vacation because I had to implement a search engine for Algorithm subject which was conducted by Madam Upuli and Sir Saliya.

We had taught several types of algorithms and data structures before this project and we had to apply those things when implementing that.

The search engine works like this. Initially we have to input the root path of the folder which we are going to search. Then the search engine should search all the files within that folder as well as the subfolders. Then it should read all the words in text files and store the words and the corresponding file paths in a suitable data structure. After finishing the storing the search engine is ready for search. Then the user can input a word alone or multiple words using & and | operations for search. then the search engine should return the corresponding file paths as the output.

At the beginning the task was not easy to achieve because I didn’t even knew how to differentiate text files and directories. So I had to work hard to fulfill this task.

I choose java to implement the search engine and these are my classes

I implemented my own vector class which have method

putValue( String data)-store data

getvalue(int index)-get the value according to the index

appendValue(String data)- append data to an existing row

to do this I had to implement an dynamic array.

This is the code for creating a dynamic array which is send by one of my friends.

1)first create a fixed array(say array[100]).

2)write a logical condition in the code, if the fixed array is full , create a new array(say tempArray) with a greater size (you can increment the fixed array size with some value).

3)then use the following method to copy the object references in the fist array to the new one.

System.arraycopy(,0,,0,) // here two zeros are to specify beginning position.

4)then make the new one referenced by the fist array name.
ex: array=tempArray;

Then I used the vector class to implement the hash table class. This has methods called

insertWord(String skey,String data)-insert the word and the path to the hash table

getData(String searchKey)-get the path of the word

checkData(String path,String data)- check the existence of the word before storing it.

In this class we use a 2D object array of vectors and after we pass the word and the path to the HashTable it converts the word to a hash value by using the inbuilt method hashCode(). Then calculate the key by

key=hashvalue%100

and key is the row of the object array. Then the class search the pre existence of that hash value in that vector and if not store the hashcode in the first column and store the path in the second column. If the hash value is already exists in that vector it appends the path of the word to the existing path.

Then the most interesting part was the searching the files in folders. And I used fileFilters to Identify the folders and the files in an given root folder.

A link for learn fileFilter

I used all these tools and finished my project successfully. I hope you can use try this so you can improve your programming skills. Keep cording ,it will train you to be an outstanding coder.

XML

2009-05-16T21:53:00.000-07:00

Why people prefer XML

XML stands for Extensible Markup Language and that was design to transfer and store data. But we can display data in XML by using style sheets (CSS, XSLT). Not like in HTML the tags are not predefined in XML and you must define your own tags as you wish.

Ex:

A good programming practice is naming tags related to the information it carries, so another developer can easily identify tags.

Another huge advantage is easy to work with data bases and computer systems. In other cases they sore data in data bases as incompatible types. But XML stores data in text format and it makes easy when storing because format is independent by software and hardware variations. The data in Xml files are send using XML DOM (Document Object Model) objects.

Try to learn XML. It will reduce your stuff.

This is a great tuutorial for beginers. http://www.w3schools.com/xml/xml_whatis.asp

Funny programs

2009-05-16T21:47:00.001-07:00

Before six months ago one of my friends came to ask some basic things in java from me. I taught him some basic things on functions and loops and I showed him how to print a triangle from stars like this.

Then I asked him that can you write a code which gives an output like this, and I went to bring a cup of tea.

When I came back he showed me the code which wrote from bottom to top.