WEET-IT Alpha version – Demo

3 Comments

it’s WEET-IT Again , as mentioned in the previous post WEET-IT  is a Semantic knowledge engine  based on semantic web and open linked data that supports different use cases like Question answering , Keyword search , Comparisons and Relations our project achieved very promising results compared to the benchmarks of “Aqua log” and “Freya”  projects – the top places projects in the QALD-1 workshop depending on their evaluation datasets .

this demo demonstrates the first Question answering algorithm  of weet-it as well as the comparison  and Relation algorithms .

the answers are Full profiles in case of one answer like “what is the capital of Egypt ? ” the answer would be one entity which is Cairo
the full profile will contain the full abstract description of the entity and the high ranked properties for that entity , similar related entities grabbed from the data graph

some answers would be a mini profile  , this is the case when the answer would be one entity or two , maybe three let’s say it’s a short description with one or two of the most relevant property to that entity

finally in case of keyword search when answers number is large we prefer to preview the micro profile which contains only the picture and a little part of the abstract .

some results would be comparison tables in case of queries like ” compare between messi and beckham ”

or relation graphs previewing relations between two entities or more like : ” what is the relation between messi , ronaldinho , xavi”  the answer most probably would be that they are fooballers , played for barcelona and other relations found in the graph .

enough of talking ,, here’s the demo : please watch in HD

we are searching for funding for hosting on larger servers and expansions issues , for interesting organizations| persons  you can contact any of the team members  @hadyelsahar @Ali_Hosny  @HatemMira @sherifkandeel @oafifi @eslamsamymeg
as well feedback , offers & interaction is very welcomed

WEET-IT the semantic knowledge engine , a 9 months story

2 Comments

9Months Ago .. i was still remembering all the talks about Semantic web and Web 3.0 .. all those definitions , some were true and some were false.

it was the time i was searching for a Graduation project lots of ideas were popping out and lots of fears of spending a year of work in a traditional thing.

i was proceeding my internship @ CMIC lab and in fact i felt my self annoying keeping asking every one about his opinion in some ideas that popped out in my head . hoping that in a way or anohter we could  get into an idea that seemed to be very cool.

after less than a month and more than a dozen of chats & meetings and more than hundreds of articles read , the idea was clear . it was a  Semantic search engine that is capable of answering Questions in a semantic way not by Text matching answers like most of traditional Search engines do. something the user could search in with terms like ” give me all android phones with capacitive touch ” or “what is the salary of cristiano ronaldo ”  or “presidential candidates in US presidential elections 2010 ”

the idea was a little vague and it’s domains of search were unclear and unknown ,  this  one day i was asking a Developer about some  keywords i could search about in the domain . then he told me an answer that seemed to be off the topic at the first time , however it was very helpful afterwards . ” why not searching in the domain of semantic web , it’s a fresh research domain that alot of people are searching in”  he said , i did a little research in semantic web at that time  , but that answer made me rethink of semantic web. a little time until i found that it’s a very promising domain with a millions of dumps of semantically annotated data , disambiguation handled  that makes  every project  tasks  easy for the researchers in the domain of data mining and information retrieval . This makes me think now it would be very nice for them to shift Into  semantic web , that would end their suffer from finding corpus free from noise  in multi languages or preprocessing dumps and categorizing them .

The idea became clear and we were  3 in the team and now came the time for finding a professor to accept this project as our graduation project , it was a tiring phase  we contacted literally  11 professors  , some were uninterested and others liked the idea but didn’t want to involve in something they have no background in , until came the day before the deadline of proposal submission that time , when a 12th professor of our contact list accepted the proposal and willingly wanted to mentor our idea .

After loads and loads of exploration,  we found some promising papers  done by a  professor named kaufmann and others related to a workshop for semantic web “QALD-1”  , We had an apartment and began development  , we made a process , we had our source controller bug tracker and a little server for hosting the database server .

I don’t remember the number of meetings we had  they were nearly every day for 10 hours  less or more a day , but I’m sure we exceeded  20 meetings a month .

Long story short , yesterday was the project defense , we were presenting  ” WEET-IT ”  (know – it )   our semantic knowledge engine , weet-it  handles  natural language queries in lots of domains  , letting the using ask  natural language free questions  for example like :

  • What is the birth place of barack obama  ?
  • What is the area of Egypt  ?
  • What are the countries that the river nile flows through ?
  • What is the profession of the wife of barack obama ?
  • What is the birth place of the children of hosni mubarak ?
  • What is the position of Messi ?

Previewing answers in form of profiles for the entities returned

As well as  questions that needs special visualization other than direct answers  like

  • Compare between Egypt and United states ?
  • What is the relation between Egypt and aboutreka ?

Our bench marks were very promising relatively to the evaluation of  “aqua log ” and ” FREYA ” the top question answering systems  in the QALD  workshops for semantic web

We believe that our results are  competitive to  EVI  the famous  Question answering application .

In the next days , we will be searching for fund for hosting on servers  as well as  some expansion and refinement

Our next steps would be adding more interesting statistics and datasets to our data base as well as exploration of new algorithms to enhance some search queries , aiming to handle subjective queries like   ” what is the best android phone with capacitive touch ”   or  ” tell me what people think of saddam hussien ? ”

” presidential candidates of Egypt ? ”  , previewing answers of charts and statistics elaborating people sentiment and reviews about those products and entities

ps : we are searching for funding for hosting on larger servers and expansions issues , for interesting organizations you can contact me   or any of the team members .

sorting algorithms #2

2 Comments

and we are back 🙂 , now let’s continue with  some other sorting algorithms  if you haven’t read part one of this Blog post you can  find it here  Sorting algorithms 

selection sort O(n^2)  : 

Selection sort worst case  \mathcal{}  n^2  average case  \mathcal{}  n^2  best case   \mathcal{}  n^2  disadvantages : it doesn’t ever benefit from presorted arrays 

lets start again with a stupid slow  sorting algorithm  , it’s the selection sort   it’s a very basic algorithm even more basic than bubble sort , it’s logic like when you have a bunch of numbers  and you want to sort them , you take the smallest one  , then the second smallest one , then the 3rd smallest one and so on .

and from here comes the naming  ( selection ) , and every time you have to loop on the array to decide the smallest item  (complexity O(n) ) and you repeat this n times in order to pick  n items ,   so the overall complexity will be O( n^2 )

 

 

 

 

 

why the selection turtle is dumber than the Bubble turtle  ?   

selection sort never takes benefit from  presorted arrays or even presorted elements every time it loops on the total array to decide the minimum rest element to pick it   so if we have a presorted array  which will be the best case 
the bubble sort  will sort it in  n iterations  , while Bubble sort will have complexity of  n^2 which makes it the dumbest ever 😀  

_____________________________

Quick Sort  : 

Quicksort  best case  : \mathcal{} n \log n   average case : \mathcal{} n \log n  worst case : \mathcal{} n^2

 

 

 

 

 

 

Quick sort  , who called it Quick ?    

 dunno . maybe it was the Quickest then ?  😀 

kidding 😀 , Quick sort is one of the most efficient algorithms  it has n^2  as a worst case but it often reaches it’s worst case it’s also faster than those  nlogn algorithms in practice 
 because its inner loop can be efficiently implemented by most architectures ( array , linked list ..etc ) 

actually : Quick sort also  competes with merge sort  ,  that Quick sort could be implemented on Linked lists  and as well as that it takes a little Memory utilization needed usually ( Log(n) )  unlike merge sort wich needs  (n) extra memory when using arrays 

let’s see how does Quick sort works : 

Quick  sort Algorithm :

it is from (divide and conquer algorithms , like the  Merge sort )

  1. Divide : using a pivot ( random number selected )
  2. sort  all the numbers less than the Pivot on  before it  
  3. sort all the numbers greater than the pivot after it 
  4. then you have two arrays one all it’s element smaller than the pivot and the other larger than the pivot 
  5. repeat all those steps recursively on the two arrays 

step  2  and  3  are done as following : 

*we use two pointers , one points to the top of the array  ,and the other at the bottom  and in the middle lies the pivot  move the left pointer until you find an element greater than the pivot ( it should to to the next side )

* on the other side move the right pointer until it hit a number less than the pivot  ( it should go to the next side )

*replace the two numbers

*  repeat those steps until you reach the pivot then the left array will become less than the pivot , and the right is more than the pivot

check this video it elaborates will the using of two pointers : 

 you can check this video for further demonstration  and some code snippets :

_____________________________

 

Heap Sort :

Heapsort  best case \mathcal{} {n \log n}  average case \mathcal{} {n \log n}  worst case \mathcal{} {n \log n}

this post is supposing that you have a pre-knowlege of what’s heap if you don’t , you can check this video  or this  

Heap sort is an efficient  kind  of  Selection sort  ( yeah the Dumbest one ever ) 😀 

How ? ? 

in selection sort we used to search for the smallest number in the array  so it was a complex algorithm of complexity n^2 

but in heaps you can easily search for elements inside it .  i needs number of iterations = height of the Tree = log(n) 

so it Gives an overall complexity of  O(nlog(n)) 

this is how it works 

  • Step 1: Build a heap
  • Step 2: removeMin()

the minimum item in the heap will be always the Root of the Heap so we will always remove it and reconstruct the heap again and it will be super easy when you are dealing with a Heap class that have remove Function

removing item from the heap

when you remove the root of the heap you and replace it with the last element in the heap

then you compare it with it’s two children , and replace the root with the minimum of it’s two children

then repeate those steps with the replaced item untill the root is smaller than it’s children

( if you don’t Get this , it’s recommended to revise the Heap removal )

the worst case here is when you compare the item to the Bottom element of the TREE , then you will do log(n) iterations

for removing n items you will have and overall complexity of O(nlog(n)) which is the complexity of the heap sort

comparing to the Selection sort (O(n^2)) it’s a Huge progress, although both based on the same logic

__________________________________________________________________

till now we are Done 🙂

yet let’s do some conclusions 😀
from different searching algorithms above and in the last post we found that

in the average case :

our three heroes will be the : Merge sort , Quick Sort , Heap sort

and our Loosers will be : the Bubble sort and the Selection sort and the insertion sort

in case of a preordered array :

the true heroes will be : the bubble and the Intersion sort for having a O(n) complexity

and the looser will be : the dumb turtle 😀 the Insertion sort

Finally that most of us before using the Algorithms was using the Bubble sorting algorithm to sort Arrays

so it once again will be the Hero as the Most Famous Sorting Algorithm 

______________________________________________________________

DO IT yourself  😀 

 :  

those people who do folk  algorithmic dances watch their youtube channel 

those Guys are awesome 🙂

it’s the best practice now to watch their Dances and try to map them to what you’ve  learnt on different sorting algorithms

also follow their fanpage

________________________________

and once again Credit Goes to :

  • MIT opencourse ware :  introduction to algorithms course , with charles and Eric  🙂  i recommend to watch the Full playlist  or take a look on  the full course with the slides and exercises  the from the MIT opencourseware website  here  thank you MIT 🙂
  • thanks to wikipedia for the Amazing animated Gif  images  🙂
  • Google for images of  Rabbits and Bunnies  😀