Someone asked for the individual HarappaWorld Admixture results for the Burusho and Kalash from HGDP.
In the chart below as well as in the spreadsheet, the IDs starting with "b" belong to the Burusho and those starting with "k" belong to the Kalash individuals.
You can check the spreadsheet too.
Having collected 12 datasets, I have gone through them and finally selected the samples and SNPs I want to include in my new dataset, which I'll call Reference 3.
It has 3,889 individuals and 217,957 SNPs. Since this is a South Asia focused blog, there are a total of 558 South Asians in this reference set (compared to 398 in my Reference I).
You can see the number of SNPs of various datasets which are common to 23andme version 2, 23andme version 3 and FTDNA Family Finder (Illumina chip).
The following datasets had more than 280,000 SNPs common with all three platforms and hence were included in Reference 3:
Reich et al had about 100,000 SNPs in common with 23andme (v2 & v3 intersection) and 137,000 with FTDNA, but there was not a great overlap. Only 59,000 Reich et al SNPs were present in all three platforms. Since I really wanted Reich et al data in Reference 3, I included it but the SNPs used for FTDNA comparisons won't be the same as for the 23andme comparisons.
Of the datasets I could not include, I am most disappointed about the Pan-Asian dataset since it has a good coverage of South and Southeast Asia. Unfortunately, it has only 19,000 SNPs in common with 23andme v2 and 23,000 with 23andme v3. I am going to have to do some analyses with the Pan-Asian data but it just can't be included in my Reference 3.
I am also interested in doing some analysis with the Henn et al African data with about 52,000 SNPs for personal reasons.
Xing et al has about 71,000 SNPs in common with 23andme v3, so some good work could be done with that, though I'll have to use only 23andme version 3 participants.
The information about the populations included in Reference 3 is in a spreadsheet as usual.
I am interested in North African populations due to my own heritage, so when Razib alerted me that Henn et al had a paper out about South African origins of humans and their African dataset was publicly available and included populations from all over Africa, I immediately downloaded it.
I have also been considering looking into the East Asian admixture in South Asians and Iranians in some detail to see where it originates from: Southeast Asia, Chinese/Japanese/Koreans, or the Turkic/Mongolian/Siberian populations of interior northeastern Asia. At a quick glance, Razib is correct:
The eastern Asian components are enriched among Bengalis, as you’d expect, but they’re found in different proportions among many individuals who hail from the northern fringe of South Asia more generally. It seems clear that the further west you go, the more likely the “eastern†element is going to be Turk, while the further east (and to some extent south) the more likely it is to be more southernly in provenance.
To do a better job though, it would be better to have more than the Yakut as an examplar of the Siberian component as I have done till now. Therefore, I downloaded the arctic populations dataset from Rasmussen et al.
Combining Henn et al and Rasmussen et al with my previous datasets (HapMap, HGDP, SGVP, Behar et al and Xing et al), I got 3,970 samples with a total of 1,716,031 SNPs represented, though at 99% genotyping rate it gets reduced to about 27,000 SNPs.
I did not remove any populations or individuals except for any duplicates and non-founders.
Here's the information on the populations represented in this dataset.
Now I am on the lookout for more datasets that are public, have enough SNPs in common with this set and can easily be converted into the Plink PED format. So if you know of any, let me know. May be I will have the biggest and most diverse dataset with your help.
For converting the HGDP data (from Stanford University) to Plink PED format, I used the following code.
#!/bin/bash dos2unix HGDP_FinalReport_Forward.txt dos2unix HGDP_Map.txt dos2unix SampleInformation.txt head --lines=1 HGDP_FinalReport_Forward.txt > header.txt awk '{for (i=1;i<=NF;i++) print "0",$i,"0","0"}' header.txt > hgdp_nosex.tfam sed '1d' HGDP_FinalReport_Forward.txt > HGDP_Data_NoHeader.txt sort -k 1b,1 HGDP_Data_NoHeader.txt > HGDP_Data_Sorted.txt sort -k 1b,1 HGDP_Map.txt > HGDP_Map_Sorted.txt join -j 1 HGDP_Map_Sorted.txt HGDP_Data_Sorted.txt > HGDP_compound.txt awk '{if ($2=="M") $2="MT";printf("%s %s 0 %s ",$2,$1,$3); for (i=4;i<=NF;i++) printf("%s %s ",substr($i,1,1),substr($i,2,1)); printf("\n")}' HGDP_compound.txt > hgdp.tped # Add sex info sed '1d' SampleInformation.txt > temp.txt sed '$d' temp.txt > SampleInfo_noheader.txt awk '{printf("HGDP%05d ",$1); if ($6=="m") print "1"; else if ($6=="f") print "2"; else print "0";}' SampleInfo_noheader.txt > Sample_sex.txt awk 'BEGIN { while ((getline < "Sample_sex.txt") > 0) f2array[$1] = $2} {if (f2array[$2]) print $1, $2, $3, $4, f2array[$2], "0" else print $2 "not listed in file2" > "unmatched" }' hgdp_nosex.tfam > hgdp.tfam # convert to ped plink --tfile hgdp --out hgdp --make-bed --missing-genotype - --output-missing-genotype 0 # Filter to 952 (or 940) people using the SampleInformation.txt file awk '{if ($16=="1") printf("0 HGDP%05d\n",$1);}' SampleInfo_noheader.txt > Sample_keep.txt plink --bfile hgdp --keep Sample_keep.txt --make-bed --out hgdp940 |
This (and more) could easily be done in Perl. You can look at the SPSmart code for some help along these lines.
I have removed San and Pygmy groups from my reference datasets. That meant removing 39 samples from Reference Data I and 61 samples from Reference Data II.
The presence of those groups was creating some weird effects in admixture runs at K=8,9. Basically, the ancestral components for Africans I was getting were not stable. Instead they were varying with/without different Harappa participant batches. Also, at K=10,11, there were too many Africa-only ancestral components, forcing me to run even higher values of K.
Since we are not really interested in African diversity in this project and any African admixture among South Asians is most likely to be East, West or North African instead of Pygmy or San, the removal of these groups should not have any implications for the Harappa Ancestry Project.
To make sure that the above assertion is true, I'll re-run admixture analysis for K=2-5 and update later with the results.
Combining my reference population with Xing et al data gets me 3,222 3,161 samples but with only about 23,000 SNPs after LD-pruning.
The good thing is that this dataset has 544 South Asian samples from 24 ethnic groups. So it'll be useful for some analyses despite the low number of SNPs. I'll try to run parallel analyses on my reference population and this dataset so we can compare the pros and cons of both.
UPDATE: I removed 61 pygmy and San samples.
For regular admixture analysis, I am using HapMap, HGDP, SGVP and Behar datasets with some samples removed as I wrote earlier.
For each of these datasets,
plink --bfile data --extract 23andmev2.snplist |
plink --bfile data --filter-founders |
plink --bfile data --geno 0.01 |
Then, I merged the datasets one by one. The reason for doing it one by one was that there were conflicts of strand orientation (forward or reverse) between the different datasets. If the merge operation gave an error, I had to flip those strands in one dataset and try the merge again.
plink --bfile data1 --bmerge data2.bed data2.bim data2.fam --make-bed plink --bfile data2 --flip plink.missnp --make-bed --out data2flip plink --bfile data1 --bmerge data2flip.bed data2flip.bim data2flip.fam --make-bed |
Once all the four datasets were merged, I processed the combined data file:
plink --bfile data --geno 0.01 |
plink --bfile data --indep-pairwise 50 5 0.3 plink --bfile data --extract plink.prune.in --make-bed |
This gave me a reference population of 2,693 2,654 individuals with each sample having about 186,000 SNPs. Out of these 2,693 2,654 individuals, we have a total of 398 South Asians belonging to 16 ethnic groups.
Finally, it's time to start having some fun!
UPDATE: I removed 39 Pygmy and San samples because they were causing some trouble with African ancestral components. Since we are not interested in detailed African ancestry and African admixture among South Asians is not likely to be pygmy or San, I decided it would be best to remove them.
Human Genome Diversity Project (HGDP) is the best resource for a diverse set of genomic data. It has 1050 individuals from 52 different populations.
I got the Stanford University data which has data for 660,918 SNPs from 1,043 samples. It is claimed that the forward strand is given but that turned out not to be true and I had to flip strands and make sure I didn't include any ambiguous A/T or C/G strands in my dataset.
I followed the recommendations of Rosenberg (spreadsheet) in excluding some atypical samples and relatives, leaving me with 940 samples.
I also excluded the Native American samples because we are not interested in them and they are very closely related either due to recent endogamy or ancient bottlenecks. (yeah I had the nerve to write that.)
Of the total of 876 samples, here are the numbers for our populations of interest:
| Balochi | 24 |
| Brahui | 25 |
| Burusho | 25 |
| Hazara | 22 |
| Kalash | 23 |
| Makrani | 25 |
| Pathan | 22 |
| Sindhi | 24 |
| Total South Asians | 190 |
|---|
These samples have about 541,560 SNPs in common with 23andme v2.
Recent Comments