On this page... (show)
- 1. Release of simuPOP 1.1.1 (May 13th, 2013)
- 2. Release of simuPOP 1.1.0 (Feb. 11th, 2013)
- 3. simuPOP 1.0.9 is updated to fix bugs in mutation and recombination (Dec. 21, 2012)
- 4. Release of simuPOP 1.0.9 (Dec. 6th, 2012)
- 5. Release of simuPOP 1.0.8 (Sep. 5th, 2012)
- 6. Release of simuPOP 1.0.7 (Apr. 4th, 2012)
- 7. Publish of a book about forward-time simulations and simuPOP.
- 8. Release of simuPOP 1.0.6 (Jul. 11, 2011)
- 9. Release of simuPOP 1.0.5 (Nov. 22, 2010)
- 10. Release of simuPOP 1.0.4 (Sep. 7, 2010)
- 11. Release of simuPOP 1.0.3 (Jun. 8, 2010)
- 12. Release of simuPOP 1.0.2 (Apr. 1, 2010)
- 13. Release of simuPOP 1.0.1 (Feb. 24, 2010)
- 14. Release of simuPOP 1.0.0 (Jan. 6th, 2010)
- 15. Release of simuPOP 0.9.9 -- the wait is almost over. (Dec. 10, 2009)
- 16. Release of simuPOP 0.9.8 -- it is time to upgrade. (Oct, 29, 2009)
- 17. Release of simuPOP 0.9.7. (Sep, 26, 2009)
- 18. Release of simuPOP 0.9.6. (Aug, 6, 2009)
- 19. Release of simuPOP 0.9.5. (Jun, 22, 2009)
- 20. Release of simuPOP 0.9.4. (May, 20, 2009)
- 21. Release of simuPOP 0.9.3. (Apr. 3, 2009)
- 22. Release of simuPOP 0.9.2. (Feb 20, 2009)
- 23. simuPOP 0.9.1 has been released today. (Jan 13, 2009)
- 24. Release of version 0.8.8 (Oct 25, 2008)
- 25. Release of version 0.8.7 (Aug, 29, 2008)
- 26. Release of simuPOP 0.8.6 (Jun 20, 2008)
- 27. Version 0.8.5 is released (Apr. 3, 2008)
- 28. Release of simuPOP 0.8.4 (Mar, 8, 2008)
- 29. Release of simuPOP 0.8.3 (Jan 17, 2008)
- 30. Release of simuPOP 0.8.2 (Nov, 19, 2007)
- 31. Release of simuPOP 0.8.1 (Oct 05, 2007)
- 32. Release of simuPOP 0.8.0 (Aug 4, 2007)
This page lists release notes of all official simuPOP releases. If you would like to see what has been changed since the last official release, you can check the ChangeLog of the development version of simuPOP.
This release comes with a few new features. Notably, it adds an operator
OffspringTagger to tag the index of offspring within a family, and a parameter
byName to allow merge chromosomes by name in function
Population.addLociFrom. It also allows a mutation operator to output details of mutation events using parameter
This is a minor bug-fixing release. The biggest change is a file format change that allows faster save/load operation with less memory usage. It also adds variables
Ne_tempoFS_P2 to statistics effectiveSize in order to calculate LD-based effective population size.
Two bugs that crash simuPOP for certain combination of parameters for mutation and recombination has been discovered and fixed. Because they might influence simulation results, please update to simuPOP 1.0.9b.
I am happy to release simuPOP version 1.0.9. This is a minor release with a few new features and bug fixes. In addition to the addition of PED/MAP and Phylip formats in function untils.export, this release adds support for the estimation of effective population size. This is an interesting and complex topic so please refer to relevant section in the simuPOP user's guide for details.
Due to a major bug in the handling of natural selection in simuPOP 1.0.6 and 1.0.7, all users of simuPOP should upgrade to this version.
This is a bug fixing release that fixes a bug that prevents natural selection from working correctly in diploid random mating schemes. Because of the potentially wide use of this feature, all uses of simuPOP are strongly encouraged to upgrade to this version.
Other than bug fixes, this release adds operator
Exporter and functions
importPopulation that support import from and export to formats such as
GENEPOP. I implemented import/export functions for these formats based on user recommendations. If you need support for other formats, please feel free to ask in the simuPOP mailing list.
simuPOP has been able to simulate large populations with long sequences, with a limit of 2^64 alleles (roughly the product of population size and the number of loci of each individual) that well exceeds the size of physical memory of most computers. For fast and parallel access to individual genotypes, simuPOP stores all genotypes linearly. This is efficient for most simulations, but not so for the simulation of rare variants in long genome sequences, where there can be few mutants (non-zero alleles) in a sea of wildtype alleles. Storing all wildtype alleles becomes insufficient in this case.
This is why we have worked very hard on the implementation of a module of simuPOP that stores only non-zero alleles of a population. By setting alleleType to 'mutant' (setOptions(alleleType='mutant')), simuPOP will make use of a storage mode that keeps indexes and values of only non-zero alleles and allow you to simulate, for examples, population genetics models that introduces random mutations over very long genome sequences. Unfortunately, because of the complexity of the storage, genotype access is much slower (can be 20 times slower depending on the density of alleles), and simultaneous write access from multiple threads is not allowed, which make this module a bad choice for small-scale simulations.
Another major addition to simuPOP is the introduction of a 'lineage' module. This module attaches each allele with a 'lineage' field which usually stores the ID of the individual to whom the allele is introduced. Because allelic lineage information is maintained during evolution, this module allows the tracking the source of mutants, and differentiating between alleles that share the same state (identity by state) and ancestry (identity by decent), which can be quite useful in certain applications. The lineage module was contributed by Michael Spiegel from UNC, who surprised me with a big patch that took me days to digest.
There are some other new features. For example, a dedicated Mitochondrial chromosome type has been introduced so that operators could handle such chromosomes in an informed way. The Stat operator now accepts parameters numOfSegSites and numOfMutants that can be used to count the number of segregating sites of genotypes of all or selected individuals, and number of mutants across a region. This operator can now also calculate Fst and Gst values for mitochondrial chromosomes and haploid populations. There are also new functions Individual.mutants() and Population.mutants() that allow you to iterate through all mutants of individuals or populations, which can be more efficient than checking all genotypes. A complete list of new features and bug fixes can be found from the ChangeLog page of the simuPOP website.
As usual, thank you for using simuPOP, and enjoy your simulations!
After several years of hard work, I am glad to announce that a monograph entitled Forward-time population genetics simulations, Methods, Implementation, and Applications has been published by Wiley & Sons Inc. This is the first book that systematically describes the methodology, implementation and applications of forward-time population genetics simulations. All examples in this book are implemented by simuPOP. I will create a special section in the simuPOP cookbook to include source code of all examples. If you have used simuPOP, please consider buying this book to support the development of simuPOP.
After more than half a year of hard work, I am glad to release version 1.0.6 of simuPOP. With less pressure on the implementation of new features, I took time to clean up the code, thoroughly test simuPOP for compatibility and performance for 64-bit operating systems, and most importantly, parallelized simuPOP using openMP. A CS student from the University of Houston has done a great job in testing and parallelizing simuPOP. This student will graduate soon so you can email me if you are looking for a good scientific programmer.
I am glad to release simuPOP 1.0.5, which is a maintenance release of the 1.0.x series. Because this release fixes two crash bugs, all users are recommended to upgrade.
The most useful feature of this release is the support of parameter subPops in during mating operators. This feature can be used to apply different during-mating operators to offspring according to their properties. For example, sex-specific selection can be implemented by applying different selection operators to virtual subpopulations defined by individual sex; and applying an operator to a virtual subpopulation with affected individuals will effectively discard all unaffected individuals. An operator DiscardIf has been provided to filter offspring according to subpopulations, an expression, or a user-defined function. Please refer to simuPOP ChangeLog for a complete list of bug fixes and new features.
With the stabilization of the simuPOP interface, I will pay more attention to the performance of simuPOP. Because many parts of simuPOP can be parallelized, I plan to use openMP and other techniques to improve the efficiency of simuPOP. If your script has been running slowly due to the use of some particular operators, please feel free to let me know.
I am glad to announce the release simuPOP 1.0.4. This is a maintenance release of the simuPOP 1.x series. Except for minor adjustments to corner cases, this release is expected to be fully compatible with previous versions of simuPOP. Please report to the simuPOP mailinglist if your existing scripts fail to execute under this release. Because this release fixes a major bug in the simuPOP core, all users are recommended to upgrade.
The biggest change of this release is the support for Python 2.7 and 3.1. Because Python 3 is incompatible with the Python 2.x series, it has been challenging to make simuPOP compatible with both versions of Python. Although these changes are largely unseen to users, you will notice that all examples in the simuPOP user's guide are written in a style that is compatible with both versions of Python. For example,
list(range(5)) is used instead of
print(a) is used instead of
print a. Because
print(a) is interpreted as printing a tuple with a single element a, it yields the same results in both Python 2.x and Python 3.x. Unfortunately, because
rpy have not been ported to Python 3, features related to these two modules are not yet available. This basically means that you can not use wxPython-based parameter input dialog (the
tkinter version should be working well), and plotting operators in module
The simuOpt module has been revised to make it easier to use. For example,
'type' has been introduced to replace
'allowedTypes' so that you can use strings such as 'integers' to specify more complex types;
'validator' (renamed from
'validate') can now take expressions such as
'len(par1) == len(par2)' to perform cross-parameter validation; and 'name' have been used to replace
'longarg'. In addition, a new
'--gui=batch' mode can be used to execute a script without specifying a large number of default parameters. Despite all these changes, your existing parameter specification dictionaries should still work as expected.
Among many new features, the ability to use subpopulation and virtual subpopulation name is most interesting. For example, you can now use
subPops=[1, 'Male'] to specify a virtual subpopulation with name
'Male' (the first vsp defined by a
SexSplitter). This feature should make your code more readable, and easier to maintain if you have a complex splitter with many VSPs.
In order to support simulation scenarios for specific applications, a new module
simuPOP.sandbox has been introduced. This module is intended to contain experimental functions and operators that might or might not be included in future versions of simuPOP. For example, if you have implemented a particular mating scheme for certain animal populations, it might be a good idea to add it to this module so that it can be distributed with simuPOP. Note that backward compatibility is not guaranteed for this module so scripts that make use of this module might be locked to a particular version of simuPOP.
I am happy to announce the release of simuPOP 1.0.3, which is a maintenance release with a number of small features. For example, a new option
GLOBAL_SEQUENCE_OF_SEX has been added to parameter
sexMode of parent choosers so that offspring sex can be controlled across families (e.g. same number of MALE and FEMALE individuals regardless of number of offspring produced at each mating scheme); A new parents chooser named
CombinedParentsChooser can now be used to combine the parents selected by two parent choosers; and a parameter
size can be used in
simuOpt.valueListOf to check the size of the input list. Please refer to the simuPOP ChangeLog for a complete list of new features.
The most useful feature, however, is that parameter
loci in all functions and operators now accepts loci names in addition to loci indexes. Because loci names will not be changed with the addition or removal of other loci, this feature makes it easy to locate loci in simulations where loci are added or removed dynamically, which is especially important for mating schemes or operators that are expected to be applied to populations with different numbers of loci. This feature was planned for a later version of simuPOP but a user request brought it to simuPOP 1.0.3. If you need any feature for your simulation, please do not hesitate to ask.
I have just released simuPOP version 1.0.2. This is a maintenance release of simuPOP 1.0.1 with a few new features. The most useful feature is the addition of parameter haplotypes to operator InitGenotype, which allows initialization of individual genotypes by haplotypes. This release also reimplemented the way how random number of offspring (parameter numOffspring of mating schemes) is generated. Please check simuPOP change log for more information.
I am happy to release simuPOP 1.0.1, which is a bug fixing release for simuPOP 1.0.0 with some new features for pedigree handling. Although none of the bugs is detrimental, it is recommended that all users upgrade to this release.
Three member functions
Pedigree.save have been added to the
Pedigree class. These functions identify all ancestors or offspring of a given set of individuals and can help you study the structure of a complete pedigree. Although a
pedigree object can be saved as a population object with all structural information, a new format has been introduced to save a pedigree in human readable format (similar to a .ped file used by many genetic analysis software). Because it can be troublesome to create a pedigree object with thousands of ancestral generations, the same file can be created using a
PedigreeTagger operator during evolution. This format can be loaded using a new global function
A pedigree object can now be used to direct the evolution of a population using a new mating scheme called
PedigreeMating. This mating scheme takes a pedigree object and evolves a population according to the parental relationships saved in a pedigree object. This makes it possible to evolve a population using pre-determined pedigree structure. Potential applications include gene-dropping, a simulation technique that assign random genotypes to leaves of a pedigree and then dropping the genotypes to other individuals according to Mendelian laws.
Among other new features, the ability to use
ALL_AVAIL in the specification of virtual subpopulations is most interesting. For example, you can now use
allIndividuals(subPops=[(ALL_AVAIL, 1)], ancGens=ALL_AVAIL) to iterate through all females in a multi-generational population if a
SexSplitter is used, regardless of the number of subpopulations in the ancestral generations. Similarly, you can use
stat(alleleFreq=0, subPops=[(1, ALL_AVAIL)]) to calculate allele frequencies in all virtual subpopulations. Please refer to the change log for a complete list of bug fixes and new features.
After more than a year's hard work, I am happy to announce the release of simuPOP 1.0.0. This release marks the end of the tumultuous 0.9.x series and the beginning of the new 1.x.x development cycle when backward compatibility will be again enforced. All simuPOP users should upgrade to this release.
Due to substantial changes to the simuPOP core and interface, this release is incompatible with previous versions of simuPOP. If you have been following the development of the 0.9.x series, this release introduces the following changes to the 0.9.9 release:
1. In order to make simuPOP conform better to Python style guide and avoid potential confusion and misuse of simuPOP, I have renamed all global functions from CapWords style to mixedCases style, and class names from mixedCases to CapWords style. This has been a very difficult decision to make because it essentially changes all names of simuPOP.
2. Parameter matingScheme has been moved from the Simulator() function to the Simulator.evolve() function, replacing the position of parameter duringOps. This change simplifies the use of during-mating operators because such operators now can only be used in the ops parameter of mating schemes.
3. A new function Population.evolve() has been introduced to evolve a population directly. The role of simulators is therefore less important than before because you only need to use a simulator if you would like to simulate multiple populations simultaneously.
4. As I have described in an earlier email to the mailing list, simuPOP now probes names of parameters of user-defined functions in order to determine what information it passes to these functions. This simplifies the use of user-defined functions because parameters such as pyOperator(offspringOnly) are no longer needed. A utility class WithArgs has been introduced to allow the use of arbitrary argument lists in user-defined functions.
There are many other small changes such as the removal of global function setRNG() (the same as getRNG().set()) and addition of function population.asPedigree (change a population to a pedigree object in place), pedigree.asPopulation (change a pedigree to a population object in place) and pedigree.identifyFamilies (separate a complete pedigree into to unrelated families). In addition, parameter ancGen has been replaced with ancGens in most functions with different usages. Please refer to the simuPOP user's guide and reference manual for details.
For users who have been using the 0.8.x series and waited patiently for the release of simuPOP 1.0, your wait is finally over. Compared to the last stable release 0.8.8, this release has substantially improved support for virtual subpopulations (VSP). For example, you can now migrate, select or mutate individuals, or calculate statistics differentially according to individual properties. This allows advanced features such as sex-biased migration, dynamic mutation rate and the calculation of statistics of arbitrary groups of individuals such as all individuals between age 20 and 40.
In order to handle pedigree structure in age-structured populations resulting from the simulation of overlapping generations, a new ID based pedigree tracking system (operators IdTagger and PedigreeTagger) has been introduced. These operators assign unique IDs to each individual and use them to track parental information during an evolutionary process. A new Pedigree class has been introduced to help you identify close and distant relatives of individuals and extract families from multi-generational or age-structured populations. All sampling functions have been rewritten and moved to a new module simuPOP.sampling.
This release also introduces numerous new features, including new operators ProductSplitter (define finer VSP from existing VSPs), InitInfo (initialize information fields), MitochondrialGenoTransmitter (a during mating operator to handle mitochondrial chromosomes), MatrixMutator (a mutation operator with arbitrary mutation matrix), MixedMutator (mixed mutation model), ContextMutator (a context-sensitive mutator), SummaryTagger (passing summary statistics from parents to offspring), new plotting operators such as ScatterPlotter, InfoPlotter, HistPlotter, QQPlotter and BoxPlotter to plot different types of plots during evolution, and new sampling functions such as drawCombinedSample to draw different types of samples (e.g. sibpairs and nuclear family) from the same population. Many details of existing functions and operators have also been changed. I have provided a quick migration guide in the ChangeLog section of the simuPOP homepage, which may help you convert your existing scripts to the latest release.
Perhaps most importantly, this release comes with a comprehensive user's guide (213 pages, with 174 examples and 23 figures) and reference manual (94 pages) that accurately document all features of simuPOP. Although simuPOP has been relatively well documented in the past, this is the first time that all advanced features such as non-random and customized operators are properly documented. To avoid killing more trees to print these documents, I have created an online version of these documents which is more user-friendly because functions and classes in the userís guide are readily linked to their references and you can download all examples. In addition, with the stabilization of the simuPOP interface, I invite you to post your functions and scripts to the simuPOP online cookbook so that others can benefit from your work.
After major revisions to the
pedigree class and the
simuPOP.sampling module, I am happy to announce the release of simuPOP 0.9.9. This release marks the completion of all new features and the 0.9.x release series. Because simuPOP 1.0 is right around the corner (I plan to release it within a month) and there are likely some interface adjustments, users are not recommended to upgrade to this release unless you are eager to test out the new sampling module.
The traditional index-based pedigrees have been replaced by a new ID based pedigree system. Because IDs are invariant across populations, the new system is much easier to work with. The resulting
simuPOP.sampling module is shorter, but more powerful than the previous one. New features include a combined sampler that allows you to draw different types of samples from a population, and a new
subPops parameter that allows you to drawn samples from selected virtual subpopulations, such as drawing cases and controls from certain age or ethnic groups. The population and pedigree classes have been modified to reflect this change, with improved
pedigree.locateRelatives, and new member functions
pedigree.individualsWithRelatives. Please refer to the simuPOP user's guide and reference manual for more details about these new features.
There are two noticeable changes to the simuPOP user interface. First, parameters and population variables
numOfFemale used in the
stat operator and some other places have been renamed to
numOfFemales as suggested by a colleague. Second, all global constant variables such as
BinomialDistribution have been changed to a
CAPITAL_LETTER_WITH_UNDERSCORE style. The previous style tends to be confusing because it is also used for global functions.
In addition, a new module simuPOP.gsl is added to simuPOP that exposes some GSL (GNU Scientific Libraries) CDF functions to the user interface. This module is provided for convenience and is in no means a complete wrapper of GSL. Because it is slow and error prone to import submodules such as
simuPOP.plotter because of their reliance on a third-party module rpy, submodules are no longer loaded by default with
'from simuPOP import *'.
I have just released simuPOP 0.9.8. With numerous enhancements in penetrance, quantitative trait and natural selection operators, all simuPOP operators have now been revisited, with full support for virtual subpopulations and complete documentation. Because version 0.9.9 will focus on pedigree and sampling related features, what you see now is pretty much the core for simuPOP 1.0. If you have been using simuPOP 0.8.8 for a relatively large project, you might want to wait for simuPOP 1.0. For all other users, it is highly recommended that you upgrade to this release. I cannot guarantee your script will work without modification under simuPOP 1.0, but this is a good time for you to test the new interface and propose any changes.
The biggest change you will notice is the removal of the stage feature of operators. Instead of specifying a stage for each operator, you now need to explicitly place your operators in parameters
finalOps of the
simulator.evolve function. To convert your script to the new version, you will need to change
finalOps, and spread your operators in parameter
ops to parameters
postOps depending on which stage they are applied to the population.
This dramatic change aims to solve a long standing problem in simuPOP in that it is difficult to see at which stage and in which order operators in parameter ops will be applied during an evolutionary process. The original design gives each operator a default stage and allows you to change it if needed. However, it is not always clear what this default stage should be. For example, if you place a
smmMutator before a
stat operator using their default stages, the
stat operator will not collect statistics for a population after mutation happens. This is because
smmMutator is by default a pre-mating operator that applies to the parental generation, and a
stat is by default a post-mating operator that applies to the offspring generation. In the new version, you are required to put operators in one or more of parameters
postOps so it is clear at which stage and in which order operators are applied.
The new version enhances support for virtual subpopulations (VSP) and overlapping generations. A new virtual splitter
productSplitter is introduced that allows you to define VSPs such as male individuals between age 20 and 40. With a new parameter
combinedSplitter, you can now combine several VSPs into one and define arbitrarily complex VSPs such as, quoting an user request, a mating VSP with males between age 20 and 40 and females between age 15 and 30. Although default names are given to VSPs, you can now give shorter names to VSPs using the names parameter of a splitter. In addition, the
cloneGenoTransmitter now copies all information fields from parents to offspring by default. This makes it much easier to use
cloneMating in overlapping generations. A new section has been added to the user's guide to describe how to use these features to simulate age structured populations and overlapping generations.
All penetrance, quantitative trait and selection operators have been revisited. The changes are too numerous to list here so please refer to the simuPOP documentation for details. The most interesting change is that simuPOP now allows natural selection through the selection of offspring, using absolute fitness values returned by a selector during the production of offspring. Although this is a slower way of doing natural selection in the standard random mating scheme, it can make a difference when you are using non-random mating or when you need to explicitly simulate the survival of offspring.
The use of demographic function has also been changed. Instead of passing subpopulation sizes to a demographic function such as
demo_func(gen, oldSize), the whole parental population is passed so you need to define you function as
demo_func(gen, pop). There are two reasons for this change. First, when there are virtual subpopulations,
oldSize might not be enough to determine subpopulation sizes of the offspring generation. Second, the new demographic function allows you to split and merge subpopulations in this function so the burden of demographic changes does not have to spread to a demographic function and operators such as
Among all new features, the new way to access individual information field is most useful. In addition to member functions
individual.setInfo, you can now access information fields as attributes of individuals so that you can use
ind.b = ind.a + 5 instead of the cumbersome
ind.setInfo(ind.info('a') + 5, 'b'). I expect this to be used extensively in simulations that involve more than a few information fields.
simuPOP takes another stride towards its 1.0 release when backward compatibility will be enforced. Before that, you are seeing another incompatible release that will likely break your existing script. Please be patient with me: all the changes are for the better and your suffer is expected to be over in a few months. Although new users are encouraged to use the latest release, existing users are not recommended to upgrade unless you need to use some of the new features described below. Note that the online documentation always reflects the latest release.
The first major change is the reorganization of utility modules.
Instead of several separated modules, simuPOP is now organized as a
simuUtil is now
simuRPy is now
simuOpt module is kept out of simuPOP but the
simuOpt.simuOpt class has been renamed to
simuOpt.simuParam due to
potential name conflicts. A completely rewritten set of classes and
functions that simulate trajectories of allele frequencies have been
simuPOP.utils. The new functions are more flexible and can
handle complicated demographic and genetic features such as population
split and merge, varying and population-specific selection pressures,
and gene gene interaction.
Another big change that will affect many existing scripts is the
initSex from operators
the sake of convenience,
initByValue have always calls
default. This has lead to some errors when multiple initialization
operators are used and some of them are sex-dependent. With the
introduction of a new operator
initInfo that initializes individual
initSex has been separated from these two
initSex now needs to be explicitly used
randomMating is used.
At a more advanced level, the way how during-mating operators are
handled has been changed slightly. The concept of '''genotype
transmitter''' and related special handling of these operators have been
removed. Consequently, all during-mating operators are handled
according to a simpler set of rules. The most visible change is that
recombinator should now be used in the
ops parameter of a mating
scheme to replace the default genotype transmitter. The existing usage
(use it in
simulator.evolve) is allowed, but will lead to reduced
The tagging operators have been revisited in this release. New
pedigreeTagger have been
introduced, and the existing operators such as
inheritTagger have been
enhanced. The introduction of
idTagger allows you to give every
individual in an evolving population an unique ID so that you can
easily identify ancestors. A function
population.indByID has been
added to look up an individual by its ID from a multi-generational
population. This feature is currently used to dump all recombination
events, but will be used in all pedigree-related features (e.g.
pedigree ascertainment) later on. Please check the use's guide for
Finally, simuPOP now accepts locus-specific allele names. That is to say, the same numeric alleles can corresponds to different alleles at different loci. This feature is introduced so that simuPOP can better handle the HapMap and other genetic datasets that uses the same 0 and 1 coding for different nucleotides at different loci. Adding locus-specific allele names make it easy, for example, to selected a subset of loci from these datasets with losing their allele information.
The stat operator is the biggest and most complicated operator of simuPOP. There were some long-standing usage problems with this operator but the solutions were far from trivial. Only until last month did I gather some time and energy to rewrite this operator, and I am happy that I did it because with the revision of this operator, I finally see some light of the 1.0 release.
The first problem is with variable
alleleFreq[loc][allele] and other
counting and frequency variables. Because it is difficult to know in
advance which alleles are in the population,
might fail. Although I have forced alleleFreq[loc] to have at least
two elements, which solves the problem for diallelic markers,
alleleFreq[loc][allele] can be invalid in other modules. To address
this problem, I have added a special dictionary type
simuPOP, which is now used for allele, genotype, haplotype and
heterozygote counts and frequencies. This dictionary returns 0 for
unrecognized keys so
alleleFreq[loc][allele] will return 0 if
specified allele does not exist in a population. Because only existing
alleles are listed,
alleleFreq[loc].keys() lists available alleles and
you can use
len(alleleFreq[loc]) to get number of alleles at locus
loc. Because of this, statistic
numOfAlleles have been removed.
The second problem is how to control which variables to output and
whether or not to calculate statistics for all subpopulations.
LD_param) have been used for each statistic but these
parameters were difficult to use. The new stat operator now has a
parameter that allows you to specify which variables to output, and
use special variable names (e.g.
alleleFreq_sp) to control whether or
not statistics should be calculated for all subpopulations. An
suffix has been introduced to allow for change of
The new stat operator now supports parameter
subPops so that you can
calculate statistics for specified (virtual) subpopulations. This
makes it easy to calculate partial statistics such as pairwise Fst.
More importantly, the use of virtual subpopulations allows you to
calculate, for example, allele frequency among cases and controls, and
association tests among only smokers.
Other than these, there has been several new statistics such as
maxOfInfo for the
calculation of basic statistics of specified information fields, new
genotype based genetic association tests (Chi-square and Armitage
trend test), and new statistics for population structure (Nei's Gst).
Perhaps more importantly, all statistics of the stat operator has now
been carefully documented with examples. Please check the relevant sections of the user's guide and reference manual for details.
There have been other interface changes. If your scripts use functions
population.ancestor(), nested lists for parameters
lociNames of function population, and parameter rep of operators,
please check the ChangeLog or updated user's guide because these
functions and parameters have been changed.
I am glad to announce the release of simuPOP 0.9.5. This release is marked by significant improvements in mutation operators. Many new mutators have been added with new features introduced to existing mutators. For example:
- Mutators can now be applied to virtual subpopulations (e.g. sex different mutation rates)
- A new matrixMutator that mutates alleles according to a mutation rate matrix.
- Specialized mutators for diallelic and nucleotide loci (snpMutator and acgtMutator).
- New mixedMutator that can apply multiple mutators with certain probabilities.
- New contextMutator that can apply different mutators according to the context of mutated allele.
- A new map in and map out feature that allows the use of mutation models when interpretation of alleles differs.
The interface of several existing mutators have also been changed. For example, the maxAllele parameter of a kamMutator is changed to k (with a different meaning), and the generalized stepwise mutation model has been merged to smmMutator. Please refer to the updated simuPOP user's guide (http://simupop.sourceforge.net/manual/build/4_userGuide.html#mutation) for details.
There has been a few other style changes. For example, the preOps and ops parameters of simulator::evolve now accept single-form inputs (ignore  when there is only one operator). In addition, parameter rate of the recombinator has been renamed to rates because this parameter accepts multiple recombination rates. This change is expected to break a lot of scripts. Please change your scripts accordingly.
varPlotter in simuRPy.py has always been the most complex Python operator in simuPOP. It has a class hierarchy of five classes that handle different types of figures. A large number of parameters are provided to customize many features of your figures. As a result, this operator has been a nightmare to test and maintain. I could not even document it properly because I was not sure which combination of parameters were supposed to work.
I am glad to report that varPlotter and the whole simuRPy module have been rewritten. Instead of providing a giant class that provides many features, the new simuRPy module provides four operators: varPlotter, scatterPlotter, infoPlotter (with histPlotter and qqPlotter as two special cases) and boxPlotter. These operators are small in size but are very powerful in the sense that you can customize every aspect of your figures using derived keyword parameters.
A derived keyword parameter is a parameter that
- Can be prefixed with a R function name to be passed to a specific R function. For example,
mar=[1,2,2,2]to R function
- Can be suffixed with an iterator name to pass different values to different replicate, subpopulation, etc. For example,
lty_rep=[1, 2, 3]will pass
lty=3to replicate 1, 2, and 3.
- Can accept an expression as its value. For example,
plot_main='!"Freq at gen %d"
% gen'will pass
main="Freq at gen 10"at generation 10.
Utilization of such a mechanism greatly simplifies the implementation of these operators, and on the other hand allows arbitrary parameters to be passed to the underlying R functions. Please visit http://simupop.sourceforge.net/manual/build/5_userGuide.html#simurpy for detailed explanations and a few examples.
I have just released the 0.9.3 version of simuPOP. This version fixes a number of bugs, including the memory leak problem with operator infoExec and infoEval. A few minor features are added to this version, most notably the ability to direct your operator output to any Python function. An example has been given in the updated user's guide.
Progress has also been made to the simuPOP documentation. All genotype transmitters are now properly documented. A few more examples are added to the online cookbook, including a function to save simuPOP population in popGene format. This makes Yaji Xu, author of this function, the first simuPOP user who contributes to the online cookbook. Thanks!
As I have announced before, the development of the 0.9.x series will not consider backward compatibility. This is again the case for today's 0.9.2 release. Briefly speaking, the following areas have been revisited, revised, and documented:
initByValue. No significant change.
- Expression and statements:
infoExec. Minor interface changes.
infoEvalcan no longer change individual information field.
- Demographic models:
migrator: Change interface, add support for virtual subpopulation, add
pyMigrator: Removed because its functionality can now be achieved by the
ByIndInfomode of a migrator.
splitSubPops: Renamed from
splitSubPop. Add support for multiple subpopulations and the ability to split by information field.
resizeSubPops: Add support for resize by proportions.
- Miscellaneous operators: Polish and add examples for
- revise how module
simuOpthandles environmental variables and commandline options. The biggest change is the replacement of
-guican take other values. This option also affects other functions that use a GUI.
simuOpt.setOptionswhich takes parameters
- rewrite function
simuOpt.getParamand make it a class
simuOpt.simuOpt. More specifically, if you have
par = simuOpt.getParam(options) if len(par) == 0: sys.exit(0) # unpack parameters (help, v1, v2, v3) = par if help: print simuOpt.usage(par, options) # run simulation simulate(v1, v2, v3)
--helpare now handled automatically) and use
par = simuOpt.simuOpt(options) if not par.getParam(): sys.exit(0) # unpack parameters v1, v2, v3 = par.asList() simulate(v1, v2, v3) # or simply use parameters as attributes of the simuOpt object. simulate(par.v1, par.v2, par.v3) # or change your interface to pass the simuOpt object directly, which # is easier to use if there are a large number of parameters. simulate(par)
- All documentations are available online at http://simupop.sourceforge.net/manual.
- A wiki-based simuPOP online cookbook is available now at http://simupop.sourceforge.net/cookbook. All recipes that were distributed with simuPOP are moved to this website.
The first thing you need to know: this version is incompatible with previous simuPOP versions! Although the overall structure and most features are unchanged, your existing script will most likely not work without modification.
The second thing you need to know: although most dramatic changes have already been made, the following simuPOP versions may not be compatible with this version. Although population, simulator and mating schemes can be considered stable, operators and utility modules will be under revision. The situation will continue until simuPOP 1.0 is released. Backward compatibility will be kept again from that version. Despite of the inconveniences, you may want to take the opportunity to express your concern about simuPOP during this time because changes will be much more difficult to incorporate after simuPOP 1.0 is released.
The conclusion: If you are happy with simuPOP 0.8.8, keep using it. If you are learning simuPOP, or if you are starting a new project, it might be a good idea to jump in this version.
There has been major interface changes and some core changes during the past few months. Many functions and parameters have been renamed, combined, or removed. During this process, I have rewritten the simuPOP reference manual and user's guide. If you are not sure what have been changed, the simuPOP user's guide is now your best source of information. The operator part of this document is not finished yet, but it already has 70+ examples that demonstrate all the features of simuPOP. In addition, the simuPOP reference manual now has accurate information about every functions for population, virtual subpopulations, simulator and mating schemes.
Version 0.9.1 introduces two new features: support for customized chromosome types such as human mitochondrial chromosomes and support for customized genotype transmitters for mating schemes. These are quite advanced features so I will not describe them here. Please refer to relevant sections in the simuPOP user's guide if you are interested.
There are numerous interface changes. Most important ones include:
- Rename population::savePopulation to save. Remove parameters compress and format.
- Support customized chromosome types.
- Rename arrGenotype to genotype, add setGenotype.
- setIndInfo and indInfo now support virtual subpopulations.
- Many other member functions have been adjusted.
- Combine parameter output and outputExpr in operators, the latter is now specified as output='!expr'.
- Remove parameter grp.
- Allow parameter rep to accept a list of replicates.
- Add parameter subPops so that (some) operators can be applied to a list of (virtual) subpopulations.
- Add initSex() operator, remove pyInit(). Add parameter initSex to operators initByFreq and initByValue.
- Remove all locus=blah shortcut for loci=[blah], allow parameter loci to take both forms of input.
- Remove operator pyIndOperator() and some taggers.
- Move case control sampler and affected sibpair sampler to the Python level. largePedigreeSample and nuclearFamilySample are temporarily unavailable.
- Remove indRange parameter in initializer and dumper, add subPops as a list of (virtual) subpopulations.
- Combine parameters newSubPopSize, newSubPopFunc to subPopSize, which can accept both types of parameters.
- Combine numOffspring, mode and numOffspringParam to numOffspring, which now accept a tuple-like input.
- Combine sexMode and sexParam to sexMode, which now accepts a tuple-like input.
- Rename pyMating to homoMating.
- Separate during-mating operators used in mating schemes as genotype transmitters.
- Allow specification of per-mating scheme during mating operators.
- Move all pre-defined homogeneous mating schemes to the Python level.
- Replace parameter 'end' to 'gen' in simulator::evolve.
- Remove the group feature and the step function.
- Add simulator::populations().
- Many functions in simuUtil are removed. They will be added later, perhaps in different form.
- simuOpt is unchanged.
- new features in simuRPy.
- hapMapUtil will be removed later.
- They are not working yet. They will be adapted to the new interface later.
I sincerely apologize for any inconvenience these changes may cause. I will do my best to help you modify your existing scripts if you send your questions to this list.
I am pleased to announce the release of simuPOP 0.8.8. This is a maintenance release without the introduction of any new feature. On the contrary, it removes several rarely used features and adjusts the interface of several functions. For example, all arrBlah() functions are now marked obsolete. These functions can be used now, but will be removed when simuPOP 1.0 is released.
Due to the change of interface, current simuPOP users are not encouraged to upgrade.
Dear simuPOP users,
I am glad to announce the release of simuPOP 0.8.7. This release fixes a few bugs found by a simuPOP user, and a long-lasting bug that prevents windows users from saving simuPOP files in gzip compressed format.
The biggest change of this release is the drop of support for binary and XML formats. They were provided for historic reasons but do not stand a place in simuPOP after the windows/gzip bug is fixed. simuPOP files saved in these obsolete formats can still be loaded. Because simuPOP now support only one file format, file extension is no longer important. The recommended extensions are now .pop for populations and .sim for simulators.
Due to the seriousness of a migration related bug, all simuPOP users are recommended to upgrade.
Dear simuPOP users,
After more than two months' intensive work, simuPOP 0.8.6 is finally released today. This release fixes a number of bugs and introduces a few important features so I strongly suggest that all simuPOP users to upgrade.
Six bugs are fixed in this release. They range from minor memory management problems to incorrect interplay between ancestral generation, information fields and migration. An upgrade is recommended because these bugs may lead to incorrect output for complicated simulation scenarios that involve migration and information fields.
A number of new features are added to simuPOP. For example, migrator now supports migration out of virtual subpopulations. An obvious application of this feature is to migrate mostly or only males from a population. Two new operators resizeSubPop amd initSex are introduced. The latter was separated from other initializers such as initFreq and is used when you only need to initialize sex.
The virtual subpopulation interface has been changed a little bit. More specifically, there is now no subpopulation-specific virtual subpopulations. If different virtual subpopulations are desired for different subpopulations, a combined virtual splitter should be used. This change is likely to break your current scripts that use virtual subpopulation but changes to your scripts should be straightforward.
Current development of simuPOP has been focusing on easy handling of pedigrees. Although simuPOP allows the evolution of multi-generation population and provides methods to record and access pedigree information, locating relatives other than direct parents have been difficult. In this release, two functions are provided to locate and trace arbitrary relative information from a multi- generation population. An infoParentsChooser is provided to mate between individuals and individuals whose indexes are specified in some information fields. A new consanguineousMating mating scheme is provided as a shortcut for infoParentsChooser and mendelianOffspringGenerator. Interested users should have a look at a new example doc/cookbook/Mating_consanguineous.py. Note that these features should be considered experimental. The whole interface may change when it stabilizes in the 0.8.7 release cycle.
There are many other small improvements such as the addition of individual::intInfo(), population::ancestor(), and the polish of tk parameter input dialog. Please refer to ChangeLog for details.
simuPOP 0.8.5 is released today. The previous versions of simuPOP has provided a flexible framework to define arbitrary nonrandom mating schemes. Despite the power of this framework, users have found it unconvenient to construct commonly used nonrandom mating schemes each time from scratch. Therefore, I have added a bunch of nonrandom mating schemes such as homogamy (using homogamousMating), polygyny and polyandry (using polygamousMating), random mating with alpha individuals (using alphaMating), and haplodiploid mating.
The last mating scheme requires native support of haplodiploid populations in the simuPOP core. Starting from this version, you can specify ploidy=Haplodiploid when you create a population. Recombinator acts differently in such a
population by copying the first copy of paternal
chromosomes directly to an offspring.
This version also addresses an concern of why simulator::evolve() has to specify ending generation, instead of how many generations to evolve. A new parameter
gen is introduced, and the use of the old parameter end is
depreciated (working but with a warning message).
I am glad to announce the release of simuPOP 0.8.4. This release fixes a few bugs in the new mating schemes introduced in 0.8.2 and 0.8.3, and adds a few new features such as a combined splitter, operators infoEval and infoExec. A few examples of nonrandom mating schemes are added to the doc/cookbook directory. They will be explained in detail in an upcoming simuPOP cookbook.
Users of advanced mating schemes are highly recommended to upgrade.
I am glad to announce the release of simuPOP 0.8.3. This version continues to improve non-random mating by adding pedigree tracing and handling capacities. For example, using parentTagger, one can now save the pedigree of the whole evolution, and replay the pedigree using a new pedigreeMating mating scheme. By creating a pedigree manually, it is easy to implement a gene-dropping algorithm using simuPOP.
Functions in hapMapUtil.py are also updated when I use them for more hapMap-related simulations.
Every major release of simuPOP introduces a major feature to simuPOP. For the 0.8.x series, simuPOP is making great progresses in the simulation of heterogeneous populations. This is achieved by the introduction of virtual subpopulations, and more flexible mating schemes. Operators are expected to work with virtual subpopulations soon.
A 'virtual subpopulation' refers to any groups of individuals, defined by a 'virtual splitter'. For example, all males, all affected individuals, or all individuals having an information field age with value 5 can form a virtual subpopulation. In contrast to subpopulations, virtual subpopulations are dynamic because individual properties are variable.
By setting virtual splitters to subpopulations, various operations can now be performed on specific virtual subpopulations. For example, it is now easy to calculate LD between affected individuals, or individuals with any other specific property. Future versions of simuPOP would allow heterogeneous mutation, recombination etc on different virtual subpopulations.
In this release, the mating schemes are reorganized to allow heterogeneous mating schemes. Two important concepts: parents chooser and offspring generator are introduced. Different parent choosers and offspring generators can be combined freely to form arbitrary mating schemes. Moreover, different mating schemes can be applied to different subpopulation, even virtual subpopulations. For example, it is now possible to apply selfing to, e.g. 20% of the parents, and random mating to the rest of the parents. Using hybrid parents choosers, it is possible to implement aged population and non-random mating schemes that consider, for example, geographic information.
Hybrid mating schemes tend to be slow because a parents chooser will be called repeatedly. If the algorithm of picking parents is complicated, it is recommended to implement this algorithm in C++. I have included a sample script scripts/demoNonRandomMating.py to demonstrate how to do this.
Please refer to the relevant section of the user's guide and reference manual for details about virtual subpopulations, various virtual splitters, new mating schemes like selfMating, pyMating and heteroMating, and varous parents choosers and offspring generators. Although a lot of testing has been done, and the performance penalty of the new features has been kept minimal, normal users are not recommended to upgrade. I expect the new features to become mature in the 0.8.x release cycle.
I am glad to announce the release of simuPOP 0.8.1. This is a regular maintenance release with a few bug fixes and improvements. The first major new feature is that information fields can now be passed to pyPenetrance and pySelector operators, which allows easy simulations of environmental factors. In addiiton, chromNames parameter is added to the constructoe of population so that chromosome names can be tracked with function chromNames(), chromName() and chromByName() functions.
I am glad to announce the release of simuPOP 0.8.0, the result of more than one year's hard work since the release of simuPOP 0.7.0.
This release contains numerous new features, optimization and bug fixes that greatly improve the use of information fields, binary modules, and the usability of simuPOP. Most importantly, with the help from a graduate research assistant, the online help system and simuPOP reference manual have been overhauled. I strongly encourage all simuPOP users to upgrade.