To cite package markovchain in publications use:

Giorgio Alfredo Spedicato. markovchain: discrete time Markov chains made easy

Corresponding BibTeX entry:

  @Manual{,
    title = {markovchain: discrete time Markov chains made easy},
    author = {Giorgio Alfredo Spedicato},
    month = {06},
    year = {2015},
    note = {R package version 0.3.0.1},
  }