To cite package markovchain in publications use:

Tae Seung Kang

Giorgio Alfredo Spedicato and Tae Seung Kang. markovchain: discrete time Markov chains made easy

Corresponding BibTeX entry:

  @Manual{,
    title = {markovchain: discrete time Markov chains made easy},
    author = {Giorgio Alfredo Spedicato},
    month = {07},
    year = {2015},
    note = {R package version 0.3.1},
  }