Skip to main content
SpringerLink
Log in

Estimating Kolmogorov Entropy from Recurrence Plots

  • Chapter
  • First Online:
Recurrence Quantification Analysis

Part of the book series: Understanding Complex Systems ((UCS))

Abstract

Kolmogorov entropy, actually an entropy rate h, has been introduced in chaos theory to characterize quantitatively the overall temporal organization of a dynamics. Several methods have been devised to turn the mathematical definition into an operational quantity that can be estimated from experimental time series. The method based on recurrence quantitative analysis (RQA) is one of the most successful. Indeed, recurrence plots (RPs) offer a trajectory-centered viewpoint circumventing the need of a complete phase space reconstruction and estimation of the invariant measure. RP-based entropy estimation methods have been developed for either discrete-state or continuous-state systems. They rely on the statistical analysis of the length of diagonal lines in the RP. For continuous-state systems, only a lower bound K 2 can be estimated. The dependence of the estimated quantity on the tunable neighborhood radius ϵ involved in constructing the RP, termed the ϵ-entropy, gives a qualitative information on the regular, chaotic or stochastic nature of the underlying dynamics. Although some caveats have to be raised about its interpretation, Kolmogorov entropy estimated from RPs offers a simple, reliable and quantitative index, all the more if it is supplemented with other characteristics of the dynamics.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 129.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. A.N. Kolmogorov, A new metric invariant of transient dynamical systems and automorphisms in lebesgue spaces (in Russian). Doklady Akademii Nauk SSSR 119, 768–771 (1958)

    MathSciNet  Google Scholar 

  2. Ya.G. Sinai, On the concept of entropy for a dynamic system. Doklady Akademii Nauk SSSR 124, 768–771 (1959)

    Google Scholar 

  3. C.E. Shannon, A mathematical theory of communication. Bell Syst. Tech. J. 27, 479–423, 623–656 (1948)

    Article  MathSciNet  Google Scholar 

  4. T.M. Cover, J.A. Thomas, Elements of Information Theory (Wiley, New York, 1991)

    Book  MATH  Google Scholar 

  5. G. Nicolis, P. Gaspard, Toward a probabilistic approach to complex systems. Chaos Solitons Fractals 4, 41–57 (1994)

    Article  ADS  MATH  MathSciNet  Google Scholar 

  6. A. Lesne, Shannon entropy: a rigorous mathematical notion at the crossroads between probability, information theory, dynamical systems and statistical physics. Math. Struct. Comput. Sci. 24, e240311 (2014)

    Article  MathSciNet  Google Scholar 

  7. W. Ebeling, T. Pöschel, Entropy and long-range correlations in literary English. Europhys. Lett. 26, 241–246 (1994)

    Article  ADS  Google Scholar 

  8. H. Herzel, W. Ebeling, A.O. Schmitt, Entropies of biosequences: the role of repeats. Phys. Rev. E 50, 5061–5071 (1994)

    Article  ADS  Google Scholar 

  9. C.K. Peng, S.V. Buldyrev, A.L. Goldberger, S. Havlin, M. Simons, H.E. Stanley, Finite-size effects on long-range correlations: implications for analyzing DNA sequences. Phys. Rev. E 47, 3730–3733 (1993)

    Article  ADS  Google Scholar 

  10. M.P Paulus, M.A. Geyer, L.H. Gold, A.J. Mandell, Application of entropy measures derived from ergodic theory of dynamical systems to rat locomotor behavior. Proc. Natl. Acad. Sci. USA 87, 723–727 (1990)

    Google Scholar 

  11. P. Faure, H. Neumeister, D.S. Faber, H. Korn, Symbolic analysis of swimming trajectories reveals scale invariance and provides model for fish locomotion. Fractals 11, 233–243 (2003)

    Article  MathSciNet  Google Scholar 

  12. K. Doba, L. Pezard, A. Lesne, V. Christophe, J.L. Nandrino, Dynamics of emotional expression in autobiographical speech of patients with anorexia nervosa. Psychol. Rep. 101, 237–249 (2007)

    Google Scholar 

  13. K. Doba, J.L. Nandrino, A. Lesne, J. Vignau, L. Pezard, Organization of the narrative components in the autobiographical speech of anorexic patients: a statistical and non-linear dynamical analysis. New Ideas Psychol. 26, 295–308 (2008)

    Article  Google Scholar 

  14. S.P. Strong, R.B. Koberle, R.R. de Ruyter van Steveninck, W. Bialek, Entropy and information in neural spike trains. Phys. Rev. Lett. 80, 197–200 (1998)

    Google Scholar 

  15. J.M. Amigo, J. Szczepanski, E. Wajnryb, M.V. Sanchez-Vives, Estimating the entropy rate of spike trains via Lempel-Ziv complexity. Neural Comput. 16, 717–736 (2004)

    Article  MATH  Google Scholar 

  16. J.P. Eckmann, S. Kamphorst, D. Ruelle, Recurrence plots of dynamical systems. Europhys. Lett. 5, 973–977 (1987)

    Article  ADS  Google Scholar 

  17. L.L. Trulla, A. Giuliani, J.P Zbilut, C.L. Webber, Jr., Recurrence quantification analysis of the logistic equation with transients. Phys. Lett. A 223, 255–260 (1996)

    Google Scholar 

  18. C.L Webber, J.P. Zbilut, Dynamical assessment of physiological systems and states using recurrence plot strategies. J. Appl. Physiol. 76, 965–973 (1994)

    Google Scholar 

  19. C.L Webber, J.P. Zbilut, Recurrence quantifications: feature extractions from recurrence plots. Int. J. Bifurcat. Chaos 17, 3467–3475 (2007)

    Google Scholar 

  20. N. Marwan, M.C. Romano, M. Thiel, J. Kurths, Recurrence plots for the analysis of complex systems. Phys. Rep. 438, 237–329 (2007)

    Article  ADS  MathSciNet  Google Scholar 

  21. H. Kantz, T. Schreiber, Nonlinear Time Series Analysis (Cambridge University Press, Cambridge, 1997)

    MATH  Google Scholar 

  22. M. Thiel, M.C. Romano, P.L. Read, J. Kurths, Estimation of dynamical invariants without embedding by recurrence plots. Chaos 14, 234–243 (2004)

    Article  ADS  MATH  MathSciNet  Google Scholar 

  23. C.S. Daw, C.E.A. Finney, E.R. Tracy, A review of symbolic analysis of experimental data. Rev. Sci. Instrum. 74, 916–930 (2003)

    Article  ADS  Google Scholar 

  24. D Lind, B. Marcus, An Introduction to Symbolic Dynamics and Coding. (Cambridge University Press, Cambridge, 1995)

    Google Scholar 

  25. J. Guckenheimer, P. Holmes, Nonlinear Oscillations, Dynamical Systems, and Bifurcations of Vector Fields. (Springer, Berlin, 1983)

    Google Scholar 

  26. R.L. Davidchack, Y.C. Lai, E.M. Bollt, M. Dhamala, Estimating generating partitions of chaotic systems by unstable periodic orbits. Phys. Rev. E 61, 1353–1356 (2000)

    Article  ADS  Google Scholar 

  27. E.M. Bollt, T. Stanford, Y.C. Lai, K. Zyczkowski, Validity of threshold-crossing analysis of symbolic dynamics from chaotic time series. Phys. Rev. Lett. 85, 3524–3527 (2000)

    Article  ADS  Google Scholar 

  28. E.M. Bollt, T. Stanford, Y.C. Lai, K. Zyczkowski, What symbolic dynamics do we get with a misplaced partition? On the validity of threshold crossings analysis of chaotic time-series. Physica D, 154, 259–286 (2001)

    MATH  MathSciNet  Google Scholar 

  29. W. Ebeling, G. Nicolis, Word frequency and entropy of symbolic sequences: a dynamical perspective. Chaos, Solitons Fractals 2, 635–650 (1992)

    Article  ADS  MATH  MathSciNet  Google Scholar 

  30. H. Herzel, I. Grosse, Measuring correlations in symbol sequences. Physica A 216, 518–542 (1995)

    Article  ADS  MathSciNet  Google Scholar 

  31. R. Badii, A. Politi, Thermodynamics and complexity of cellular automata. Phys. Rev. Lett. 78, 444–447 (1997)

    Article  ADS  Google Scholar 

  32. P. Castiglione, M. Falcioni, A. Lesne, A. Vulpiani, Chaos and Coarse-Graining in Statistical Mechanics (Cambridge University Press, Cambridge, 2008)

    Book  MATH  Google Scholar 

  33. J. Kurths, A. Voss, A. Witt, P. Saparin, H.J. Kleiner, N. Wessel, Quantitative analysis of heart rate variability. Chaos 5, 88–94 (1995)

    Article  ADS  Google Scholar 

  34. T. Schürmann, P. Grassberger, Entropy estimation of symbol sequences. Chaos 6, 414–427 (1996)

    Article  ADS  MATH  MathSciNet  Google Scholar 

  35. A. Lesne, J.L. Blanc, L. Pezard, Entropy estimation of very short symbolic sequences. Physical Review E 79, 046208 (2009)

    Article  ADS  MathSciNet  Google Scholar 

  36. A. Lempel, J. Ziv, On the complexity of finite sequences. IEEE Trans. Inf. Theory 22, 75–81 (1976)

    Article  MATH  MathSciNet  Google Scholar 

  37. J. Ziv, A. Lempel, A universal algorithm for sequential data compression. IEEE Trans. Inf. Theory 23, 337–343 (1977)

    Article  MATH  MathSciNet  Google Scholar 

  38. J. Ziv, A. Lempel, Compression of individual sequences by variable rate coding. IEEE Trans. Inf. Theory 24, 530–536 (1978)

    Article  MATH  MathSciNet  Google Scholar 

  39. P. Faure, A. Lesne, Recurrence plots for symbolic sequences. Int. J. Bifurcat. Chaos 20, 1731–1749 (2010)

    Article  MathSciNet  Google Scholar 

  40. L Breiman, The individual ergodic theorem of information theory. Ann. Math. Stat. 28, 809–811 (1957)

    Google Scholar 

  41. B. McMillan, The basic theorems of information theory. Ann. Math. Stat. 24, 196–219 (1953)

    Article  MATH  MathSciNet  Google Scholar 

  42. P. Grassberger, I. Procaccia, Computing the Kolmogorov entropy from a chaotic signal. Phys. Rev. A 28, 2591–2593 (1983)

    Article  ADS  Google Scholar 

  43. A Cohen, I. Procaccia, Computing the Kolmogorov entropy from time signals of dissipative and conservative dynamical systems. Phys. Rev. A 31, 1872–1882 (1985)

    Google Scholar 

  44. J.P. Eckmann, D. Ruelle, Ergodic theory of chaos and strange attractors. Rev. Mod. Phys. 57, 617–656 (1985)

    Article  ADS  MATH  MathSciNet  Google Scholar 

  45. P. Faure, H. Korn, A new method to estimate the Kolmogorov entropy from recurrence plots: its application to neuronal signals. Physica D 122, 265–279 (1998)

    Article  ADS  Google Scholar 

  46. M. Thiel, M.C. Romano, J. Kurths, R. Meucci, E. Allaria, T. Arecchi, Influence of observational noise on the recurrence quantification analysis. Physica D 171, 138 (2002)

    Article  ADS  MATH  MathSciNet  Google Scholar 

  47. C. Letellier, Estimating the Shannon entropy: recurrence plots versus symbolic dynamics. Phys. Rev. Lett. 96, 254102 (2006)

    Article  ADS  Google Scholar 

  48. P. Gaspard, X.J. Wang, Noise, chaos, and (ϵ, τ)-entropy per unit time. Phys. Rep. 235, 291 (1993)

    Google Scholar 

  49. M.C. Casdagli, Recurrence plots revisited. Physica D 108, 12–44 (1997)

    Article  ADS  MATH  MathSciNet  Google Scholar 

  50. M.S. Baptista, E.J. Ngamga, P.R.F. Pinto, M. Brito, J. Kurths, Kolmogorov-Sinai entropy from recurrence times. Phys. Lett. A 374, 1135–1140 (2010)

    Article  ADS  MATH  Google Scholar 

  51. A.D. Wyner, J. Ziv, Some asymptotic properties of the entropy of a stationary ergodic data source with applications to data compression. IEEE Trans. Inf. Theory 35, 1250–1258 (1989)

    Article  MATH  MathSciNet  Google Scholar 

  52. E.J. Ngamga, D.V. Senthilkumar, A. Prasad, P. Parmananda, N. Marwan, J. Kurths, Distinguishing dynamics using recurrence-time statistics. Phys. Rev. E 85, 026217 (2012)

    Article  ADS  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. CNRS UMR 7102, Neurobiologie des Processus Adaptatifs, Université Pierre et Marie Curie-Paris 6, 4 place Jussieu, 75252, Paris Cedex 05, France

    Philippe Faure

  2. CNRS UMR 7600, Laboratoire de Physique Théorique de la Matière Condensée, Université Pierre et Marie Curie-Paris 6, 4 place Jussieu, 75252, Paris Cedex 05, France

    Annick Lesne

Authors
  1. Philippe Faure
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Annick Lesne
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Philippe Faure .

Editor information

Editors and Affiliations

  1. Department of Cell and Molecular Physiology, Loyola University, Maywood, Illinois, USA

    Charles L. Webber, Jr.

  2. Transdisciplinary Concepts and Methods, Potsdam Institute for Climate Research Research Domain, Potsdam, Germany

    Norbert Marwan

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Faure, P., Lesne, A. (2015). Estimating Kolmogorov Entropy from Recurrence Plots. In: Webber, Jr., C., Marwan, N. (eds) Recurrence Quantification Analysis. Understanding Complex Systems. Springer, Cham. https://doi.org/10.1007/978-3-319-07155-8_2

Download citation

Publish with us

Policies and ethics

Search

Navigation