In this paper, we propose continuous time Bayes’classifier to recognize hand activities which is robust solution to both spatial and temporal uncertainty. In our proposed method, conditional probabilities of temporal events can be explicitly represented by defining temporal nodes in a Bayes’ classifier. Utilizing these nodes, both temporal and spatial pattern can be simultaneously inferred in a unified framework. To validate our proposed method, we show the experimental results on hand gesture recognition.