To cite package markovchain in publications use:

Giorgio Alfredo Spedicato. markovchain: discrete time Markov chains made easy

Corresponding BibTeX entry:

  @Manual{,
    title = {markovchain: discrete time Markov chains made easy},
    author = {Giorgio Alfredo Spedicato},
    month = {05},
    year = {2015},
    note = {R package version 0.2.1},
  }