Statistical Independence or Conditional Random Fields

There are two topics, statistical independence and conditional random fields, either of which I'd like to talk about in the seminar. I am not sure I can finish the former slides in time. Here are the outlines of both topics and references which you may be interested in.

Statistical Independence

• Statistical Independence
  • definition of independence;
  • several concepts (moments, cumulants, moment generating functions, characteristic functions, skewness, kurtosis);
  • numerical estimation of independence.
    
• Kernel Method for Independence Test
  • the main theorem;
  • numerical estimators;
  • the choice of kernels.
    
• Applications
  • independent component analysis;
  • clustering;
  • sufficient dimensionality reduction;
  • unsupervised kernel dimensionality reduction;
  • test of the same distribution;
  • iid test
    

References

• [Bach and Jordan 2002], Kernel independent component analysis, JMLR.
• [Fukumizu et al 2004], Dimensionality reduction for supervised learning with reproducing kernel Hilbert spaces, JMLR.

An Introduction to Conditional Random Fields

• Probabilistic Graphical Models
  • Bayesian belief networks;
  • Markov random fields.
    
• Conditional Random Fields
  • naive Bayes and logistic regression;
  • hidden Markov model and linear-chain conditional random fields;
  • general CRFs and skip-chain CRFs;
  • computational problems.
    
• Several Extensions
  • maximum margin Markov networks;
  • semi-Markov CRFs;
  • Bayesian CRFs;
  • non-parametric Bayesian methods
    

References

• [Berger et al 1996], A maximum entropy approach to natural language processing, Computational Linguistics.
• [Lafferty et al 2001], Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data, ICML.
• [Taskar et al 2003], Max-margin Markov Networks, NIPS.