Ok found it myself. For those who want to know how to do it just refer this post:
What this basically means is that you have to get the data as XML from Tally and then just compress it as Zip file and the import it using the “Tally Migration” tool within ERPNext.