It's often difficult to identify the seed(s) for the hurricane of tameless thoughts posessing the potential of devastating oneself. Restricting these abstract entities (thoughts) in a finite dimensional space of meaning should be considered both unjustified and unfair to them. Rather, it's witful to use expressive tools often employed by writers like metaphors, to serve the twin purpose of introducing them and save yourself from the crime of restricting their definition by keeping the interpretetion flexible through mere comparison.
Following from previous hypothesis, tameless thoughts can be compared to a hurricane in all possible respects. A subset of these comparisons have been touched briefly with subtelety. At the present, Science hasn't matured enough to predict the occurence of the hurricane and neither it's origin. Secondly, tameless thoughts are known to arrive uninvited without a warning signal and swirles and distorts anythings that comes under it's path. It's a brutal animal of whim who gorges on whatever it likes. And lastly both culminate into a long period of silence and inactivity. And the prescribed procedure to save yourself from a hurricane is to run away from it's path rather than fighting with it to change it's course.
So the take home is 'Run Lola Run' like Forest Gump
- karan
- Firefox (new pen-name)
Monday, February 28, 2011
Tuesday, February 22, 2011
If and only if
While working on a linear algebra problem I revisited the conditional horror statement again which is ' This equation (call it X) will have a solution if and only if condition Y is true'. The normative procedure to prove such a hypothesis if
1. Show that if Y is true, X holds and if X holds Y is true.
However the solution in the textbook ( An invitation to 3D vision) proved it by showing the Y is both sufficient and necessary condition for X to be true. I could relate to it to some extent, however it perplexed my tiny mind in my tiny body and dis-balanced my appetite to study the specific proof further. Thus I thought of delving deep into this logical statement relationships and ambiguities.
First and foremost I figured out that it's not recommended to interpret 'only if' in in terms of it's meaning in English and then things started getting clear. Let consider two logical statements
P-> Karan will sing
Q-> CG1 (one who highly disregards my blog) and CG2 (one who comments 'thoo' on my blog) listens to it
If P then Q is similar to the following statements
1. P->Q (P is sufficient for Q)
2. Q is necessary for P
3. ~Q is sufficient for ~P (~ means negation)
4. ~P is sufficient for ~Q
In English the four statements can be written as
1. If CG1 and CG2 listens Karan will sing.
2. Only if Karan sings CG1 and CG2 listens. (See the irrelevance in the meaning of only if which gives a semantic meaning to necessity)
3. If CG1 and CG2 doesn't listen Karan will not sing.
4. Only if Karan will not sing, CG1 and CG2 will not listen.
This might be difficult to digest in one go, but the important point to take home is that to check if Y is necessary condition for X, it is always proved by checking for sufficiency of negation of Y for X since it holds a meaning.
Now it was easy for me to relate if and only if proofs with checking for sufficiency and necessity. An if condition amounts to checking for sufficiency and only if for necessity (negation solution in last paragraph).
I will re-iterate the example I was working on: Prove the two C will yield a single solution for space metric if and only if the axis of rotation for the rotation matrices are linearly independent. (An invitation to 3D vision, Theorem 6.9, pg
19)
Proof: Only if is check by showing that when the axis of rotation are the same, the SRK(L_1)=SRK(L_2), where L_i are Lypapunov maps for both the rotation matrices. 'If' is confirmed by showing the sufficiency of the condition (see the proof on your own)
- karan
1. Show that if Y is true, X holds and if X holds Y is true.
However the solution in the textbook ( An invitation to 3D vision) proved it by showing the Y is both sufficient and necessary condition for X to be true. I could relate to it to some extent, however it perplexed my tiny mind in my tiny body and dis-balanced my appetite to study the specific proof further. Thus I thought of delving deep into this logical statement relationships and ambiguities.
First and foremost I figured out that it's not recommended to interpret 'only if' in in terms of it's meaning in English and then things started getting clear. Let consider two logical statements
P-> Karan will sing
Q-> CG1 (one who highly disregards my blog) and CG2 (one who comments 'thoo' on my blog) listens to it
If P then Q is similar to the following statements
1. P->Q (P is sufficient for Q)
2. Q is necessary for P
3. ~Q is sufficient for ~P (~ means negation)
4. ~P is sufficient for ~Q
In English the four statements can be written as
1. If CG1 and CG2 listens Karan will sing.
2. Only if Karan sings CG1 and CG2 listens. (See the irrelevance in the meaning of only if which gives a semantic meaning to necessity)
3. If CG1 and CG2 doesn't listen Karan will not sing.
4. Only if Karan will not sing, CG1 and CG2 will not listen.
This might be difficult to digest in one go, but the important point to take home is that to check if Y is necessary condition for X, it is always proved by checking for sufficiency of negation of Y for X since it holds a meaning.
Now it was easy for me to relate if and only if proofs with checking for sufficiency and necessity. An if condition amounts to checking for sufficiency and only if for necessity (negation solution in last paragraph).
I will re-iterate the example I was working on: Prove the two C will yield a single solution for space metric if and only if the axis of rotation for the rotation matrices are linearly independent. (An invitation to 3D vision, Theorem 6.9, pg
19)
Proof: Only if is check by showing that when the axis of rotation are the same, the SRK(L_1)=SRK(L_2), where L_i are Lypapunov maps for both the rotation matrices. 'If' is confirmed by showing the sufficiency of the condition (see the proof on your own)
- karan
Saturday, February 19, 2011
Kalman Filter: Linear Predictive Filtering
Most of my comments follow from the excellent report on Kalman filtering that is quite comprehensive and analytical.
http://read.pudn.com/downloads65/ebook/232698/A%20Study%20of%20the%20Kalman%20Filter%20applied%20to%20Visual%20Tracking.pdf
What is the fist application arises in your mind on hearing the term 'Kalman Filters'. To a vision or even signal processing enthusiastic is object tracking be it in the field of images, navigation systems, radar etc. This fact made me inquisitive to know about Kalman Filters and how does it help in these applications.
On signal processing step, it is a linear predictive filter, which in layman terms is that it assumes a linear relationship between future and past observations. An example is the x(t) and y(t) coordinates of a moving object at time t may be related to x(t+1) and y(t+1) by a simple linear model. Alongwith linear relationships, Kalman filter also assumes a noise model- zero mean Gaussian in most cases, whose parameters are updated on each observations.
I found it easy to understand the context of Kalman filters in this report than Wikipedia, may be because it explains it in the context of object tracking. It was also interesting to see how HMM can be linked to Kalman filters. Ok so whats HMM might be next question waiting for us.
HMM also known as hidden Markov models (don't ask me why I capitalized M) allows one to link a sequence of observations (essentially a RV). The way they differ from Markov models is that they encompass a hidden layer in addition to observation layer. Lets take the same example, in case of a simple Markov model (not hidden) you can define the state (x and y coordinates) transitions in terms of probabilities, like lets's say probability of a car going from x=1,y=0 to x=10,y=0. The addition that HMM brings in is one is also able to take into account hidden variables like acceleration, velocity etc. into account. This hidden layer in the case of Kalman filtering is related to observation layer by a linear transform (to make things simple again).
So how does a Kalman Filter works. It's not possible for me to do away with some basic notation. x(t) represents the current hidden state and y(t) the current observation.
1. Time-update step: Use x(t) to predict x(t+1) using equation of the form x(t)=Ax(t+1) and noise correlation matrix and further estimate y(t+1) [again linearly related].
2. Once observation is available, which in the case of tracking is obtained by comparing features in a window, use it to update x(t+1) and noise correlation matrix.
In Kalman filters since we don't have any prior knowledge about noise correlation matrices, they are an important determinant of it's performance.
The way to see all this in a somewhat clear picture is to understand how Kalman filtering and a features tracking algorithm fit into a picture. Kalman filtering is a predictive algorithm that sits on the top of feature tracking algorithm that returns the location where the feature migrated. Kalman filtering can reduce or augment this search space for feature location by predicting it's new position. This approach may not seem to be clicking for applications like face tracking in front of monitor, however they are relevant to cases when one expects high variations in object position or frame rate is slow and the task may even require to move the camera (tracking an airplane is a good example).
I would refer the readers to have a look at the conclusion and disucssion section of the report for some insights.
Thanks
- Karan
http://read.pudn.com/downloads65/ebook/232698/A%20Study%20of%20the%20Kalman%20Filter%20applied%20to%20Visual%20Tracking.pdf
What is the fist application arises in your mind on hearing the term 'Kalman Filters'. To a vision or even signal processing enthusiastic is object tracking be it in the field of images, navigation systems, radar etc. This fact made me inquisitive to know about Kalman Filters and how does it help in these applications.
On signal processing step, it is a linear predictive filter, which in layman terms is that it assumes a linear relationship between future and past observations. An example is the x(t) and y(t) coordinates of a moving object at time t may be related to x(t+1) and y(t+1) by a simple linear model. Alongwith linear relationships, Kalman filter also assumes a noise model- zero mean Gaussian in most cases, whose parameters are updated on each observations.
I found it easy to understand the context of Kalman filters in this report than Wikipedia, may be because it explains it in the context of object tracking. It was also interesting to see how HMM can be linked to Kalman filters. Ok so whats HMM might be next question waiting for us.
HMM also known as hidden Markov models (don't ask me why I capitalized M) allows one to link a sequence of observations (essentially a RV). The way they differ from Markov models is that they encompass a hidden layer in addition to observation layer. Lets take the same example, in case of a simple Markov model (not hidden) you can define the state (x and y coordinates) transitions in terms of probabilities, like lets's say probability of a car going from x=1,y=0 to x=10,y=0. The addition that HMM brings in is one is also able to take into account hidden variables like acceleration, velocity etc. into account. This hidden layer in the case of Kalman filtering is related to observation layer by a linear transform (to make things simple again).
So how does a Kalman Filter works. It's not possible for me to do away with some basic notation. x(t) represents the current hidden state and y(t) the current observation.
1. Time-update step: Use x(t) to predict x(t+1) using equation of the form x(t)=Ax(t+1) and noise correlation matrix and further estimate y(t+1) [again linearly related].
2. Once observation is available, which in the case of tracking is obtained by comparing features in a window, use it to update x(t+1) and noise correlation matrix.
In Kalman filters since we don't have any prior knowledge about noise correlation matrices, they are an important determinant of it's performance.
The way to see all this in a somewhat clear picture is to understand how Kalman filtering and a features tracking algorithm fit into a picture. Kalman filtering is a predictive algorithm that sits on the top of feature tracking algorithm that returns the location where the feature migrated. Kalman filtering can reduce or augment this search space for feature location by predicting it's new position. This approach may not seem to be clicking for applications like face tracking in front of monitor, however they are relevant to cases when one expects high variations in object position or frame rate is slow and the task may even require to move the camera (tracking an airplane is a good example).
I would refer the readers to have a look at the conclusion and disucssion section of the report for some insights.
Thanks
- Karan
Friday, February 18, 2011
Scale Space Representation
This would be my first technical article and since I don't have much time to write this, I will be primarily writing some stuff from my notes.
The idea of scale space representation was initiated by Linderberg currently a Professor at KTH, Sweden. It provides a naive multi-scale representation for signals (images in this case) using a single parameters apart from regular variables. It can be expressed as L(x,y,σ)=I(x,y)*G(x,y,σ) and it has been shown by obscure derivations by Linderberg that the only suitable kernel that mkes this representation plausible (linear in some sense) is Gaussian.
Intuitively different versions of L exhibit different extent of blurring by different σ. This can be interpreted as each level preserving only certain structures that reduces as we reach for higher σ. This idea salient feature of this representation which allows one to talk about strucutres that exist or are more dominant at certain scales, making it possible to extract them.
This representation has formed the basis of many influential works in interest point detectors, the most famous being SIFT. SIFT features- interest points exploit the fact that the local extrema of the Laplacian of Gaussian operation in scale space contains salient/stable/prominent/scale-invariant points and further uses DOG (Difference of Gaussian) approximation to identify these points. This entire theory seems be tractable owing to the beautiful properties of Gaussian, it's derivative and being able to relate these in context of image and underlying structures.
Reads
1. Lowe paper on SIFT
2. Linderberg's book- Scale-space theory in computer vision and may be his paper
3. Whole line of literature on interest points that in someway address this theory
More to follow sometime soon
- Karan
The idea of scale space representation was initiated by Linderberg currently a Professor at KTH, Sweden. It provides a naive multi-scale representation for signals (images in this case) using a single parameters apart from regular variables. It can be expressed as L(x,y,σ)=I(x,y)*G(x,y,σ) and it has been shown by obscure derivations by Linderberg that the only suitable kernel that mkes this representation plausible (linear in some sense) is Gaussian.
Intuitively different versions of L exhibit different extent of blurring by different σ. This can be interpreted as each level preserving only certain structures that reduces as we reach for higher σ. This idea salient feature of this representation which allows one to talk about strucutres that exist or are more dominant at certain scales, making it possible to extract them.
This representation has formed the basis of many influential works in interest point detectors, the most famous being SIFT. SIFT features- interest points exploit the fact that the local extrema of the Laplacian of Gaussian operation in scale space contains salient/stable/prominent/scale-invariant points and further uses DOG (Difference of Gaussian) approximation to identify these points. This entire theory seems be tractable owing to the beautiful properties of Gaussian, it's derivative and being able to relate these in context of image and underlying structures.
Reads
1. Lowe paper on SIFT
2. Linderberg's book- Scale-space theory in computer vision and may be his paper
3. Whole line of literature on interest points that in someway address this theory
More to follow sometime soon
- Karan
Tuesday, February 1, 2011
Human Enterprise: Thinking
Thinking exists in dichotomy- similar to two faces of a coin it can be soothing or paining, it can be constructive as well as destructive. Despite it having a negative side, thinking is indeed human's biggest enterprise or more precisely the biggest power. Mathematically it's the best extrapolation or estimator tool developed so far, which isn't surprising keeping in mind the vast potential of the brain to accomplish complex tasks.
Similar to the training an athlete requires to be able to condition his legs for running longer and faster, humans also need to train their brains. As per a recent study (featured in TED lectures), the complexity of the brain in terms of information transfer and no. of connections grows with each phase of thinking. It is surprising to note that most of the body parts wither with age, however brain is something that becomes sharper and encompasses more knowledge with age.
It is a general opinion that thinking in excess can sometimes be dangerous, particularly when it has the potential of going uncontrolled. This problem can also exist both physically (more neuron exciters than inhibitors) or psychologically. However, I feel that unless and until someone experiences this madness, he won't be able to learn to put his thoughts in the right direction. In a rephrase, it's necessary to go through bad times to be able to differentiate and understand the importance of good times.
[to be completed]
Similar to the training an athlete requires to be able to condition his legs for running longer and faster, humans also need to train their brains. As per a recent study (featured in TED lectures), the complexity of the brain in terms of information transfer and no. of connections grows with each phase of thinking. It is surprising to note that most of the body parts wither with age, however brain is something that becomes sharper and encompasses more knowledge with age.
It is a general opinion that thinking in excess can sometimes be dangerous, particularly when it has the potential of going uncontrolled. This problem can also exist both physically (more neuron exciters than inhibitors) or psychologically. However, I feel that unless and until someone experiences this madness, he won't be able to learn to put his thoughts in the right direction. In a rephrase, it's necessary to go through bad times to be able to differentiate and understand the importance of good times.
[to be completed]
Wednesday, October 13, 2010
World of probability
Chronicles of this post: I still remember when I started writing this post out of frustration at beginning of my first quarter at UCSD. Nevertheless it will be relief today to bring it to completion.
As a personal note, I am open to any suggestions in this article's technicality and articulation.
The drive behind writing this post was to help myself in building an image for different concepts in probability- which forms the background of innumerable fields in mathematics and engineering. Secondly most of these ideas are a result of reading different subjects spread across multiple texts dealing with probability. And probably this might help some people in getting the big picture.
Being interested in machine learning, I would like to leverage my limited knowledge in the field to express one of the multiple views of looking at probability. Machine learning rotates around the principle that our knowledge or data gathering capabilities are always limited or more precisely finite. The rationale behind acquiring data is to be able to leverage it to answer specific questions e.g. temperature recordings to predict the occurrence of rainfall. Lets try and understand the result of manifestation of these limitations in our observational powers by revisiting the rainfall example. Even after so many years of research in weather processes, it's impossible to predict for sure if it would rain on a certain day or not. This is again accounted to the fact that humans (alongwith technology developed) have limited knowledge of the factors responsible for rain and even with what all information we have it's not possible to make a 100% accurate decisions. Thus probability have given statisticians the tool to handle uncertainties and subsequently design optimal decision rules. Lets revisit some of the basic terminology of probability.
1. Probability
a. Introduction: I read this statement in a course book (Stark) on random processes that there were two different motivations to develop the field of probability. These views can be categorized under atheistic and theistic view. A religious person wants to understand probability since he believes that there are certain forces that are exerted by the almighty and its not possible for humans to understand them. On the other hand an atheist has somewhat same belief without assuming that there a bigger hand who is playing with these forces. An atheist is more optimistic about incapabilities of human observations as mentioned earlier.
In nutshell both ideas point to a single motivation of using probability to understand and make decision about processes /experiments that are difficult to define or formulate in a deterministic way. An example to highlight this statement is the Newton 2nd Law F=ma, whereby the relation makes it possible to view forces in a deterministic way. As additional information that might form a separate topic of discussion is that it was Einstein theory of relativity that shook the premise on which earlier laws were based. The prime focus of this theory is the fact that nothing is this world can be predicted with assurance (Uncertainty principle: shttp://en.wikipedia.org/wiki/Uncertainty_principle); in other words everything in this world is probabilistic.
A quick question that might arise in anyone's mind reading this article is if probability deals with uncertainty what utility it has in the real world. I will try to answer this question by quoting a statement by most Profs- no choice in this world is best; rather the best solution to anything is one that is optimal. This optimal solution can be found by optimizing a criteria which can be some cost function or complex equation. Hence probability is about finding an optimal solution to a problem and this assumption needs to be taken into account before using it. An example of cost function in case of a classifier that decides whether to cross road is seeking for minimum number of cars crossing the street.
b. Tools of probability: This part might seem a bit mathematical, however the intent is to provide details in KISS (Keep it Simple Stupid) framework.
Random Variable: It seems the mathematician were not quite happy with the use of the notation where probabilities are defined directly on sample space like P(rain occurring/clouds seen) . Hence for the purpose of simplifying notations, they introduced the concept of random variables that maps the sample space to a real line. Although most of the current literature provides a special definition of random variable (as written), I personally don’t understand the need to give any special distinction to RV. This can be party accounted to the reason that a random variable describes a random experiment where each outcome is associated with a certain probability that is defined by a probability density/mass function. It is also easy to understand a random variable by noting its name that says it is a variable (that can take on different values) that takes on values randomly.
A RV can be used to represent any experiment/process/phenomena in real life, say the reading of a machine can be assumed a RV that takes on multiple values. And here comes the idea of inference or optimal decision- is it possible to predict the state of machine based on the observables/reading.
Product and sum rule of probability: I read this statement in a famous book on PRML by Christopher Bishop that the entire field of probability is confined into the two rules- product and sum of probability. In other words, any operation in probability can be broken down into these two rules. Hence it is vital to recognize their importance.
Independent variables: The formulation of independent RVs is more of a tool than a concept that simplifies complex calculations and at the same time allows one to take into account information from the real world. For instance in case of two machines working independently, it is reasonable to assume that the two random variables representing their readings are independent. Two variable are said to be independent when it is not possible to make any(again any) prediction about a RV variable while other is known.
An important concept underlying the last idea is to understand why pair-wise independence doesn't imply mutual independence. Suppose there are two independent experiments where coins are being flipped. Let us name these RVs as X and Y and suppose there is another RV constructed called W that is defined as W=XY. Clearly W is independent of both X and Y individually since knowing one doesn't yield any information about W. However W is not independent of both at the same time since if we know both X and Y, W can be predicted.
c. Joint Random Variable: These are an extension of distribution of single random variables and used to represent joint experiments. In nutshell a joint experiment means to record the value of more than one RV simultaneously. Suppose Y=X+W, here f(X,Y) is a joint distribution function that can be used to estimate the probability of x and y occurring together. The only relevant information that is known to be extracted from JRM is joint moments like E[XY], that is also called correlation. One can also obtain central moments like E[(X-ux)(Y-uy)] called covariance. An interesting question that might occur to anyone is how are they important and what information they yield about the random variable.
Suppose two RV are independent then there is covariance is known to be zero. However covariance being zero can never imply two RVs being independent. This is due to the fact that correlation or covariance only yield information about the linearity between two RVs.
As a personal note, I am open to any suggestions in this article's technicality and articulation.
The drive behind writing this post was to help myself in building an image for different concepts in probability- which forms the background of innumerable fields in mathematics and engineering. Secondly most of these ideas are a result of reading different subjects spread across multiple texts dealing with probability. And probably this might help some people in getting the big picture.
Being interested in machine learning, I would like to leverage my limited knowledge in the field to express one of the multiple views of looking at probability. Machine learning rotates around the principle that our knowledge or data gathering capabilities are always limited or more precisely finite. The rationale behind acquiring data is to be able to leverage it to answer specific questions e.g. temperature recordings to predict the occurrence of rainfall. Lets try and understand the result of manifestation of these limitations in our observational powers by revisiting the rainfall example. Even after so many years of research in weather processes, it's impossible to predict for sure if it would rain on a certain day or not. This is again accounted to the fact that humans (alongwith technology developed) have limited knowledge of the factors responsible for rain and even with what all information we have it's not possible to make a 100% accurate decisions. Thus probability have given statisticians the tool to handle uncertainties and subsequently design optimal decision rules. Lets revisit some of the basic terminology of probability.
1. Probability
a. Introduction: I read this statement in a course book (Stark) on random processes that there were two different motivations to develop the field of probability. These views can be categorized under atheistic and theistic view. A religious person wants to understand probability since he believes that there are certain forces that are exerted by the almighty and its not possible for humans to understand them. On the other hand an atheist has somewhat same belief without assuming that there a bigger hand who is playing with these forces. An atheist is more optimistic about incapabilities of human observations as mentioned earlier.
In nutshell both ideas point to a single motivation of using probability to understand and make decision about processes /experiments that are difficult to define or formulate in a deterministic way. An example to highlight this statement is the Newton 2nd Law F=ma, whereby the relation makes it possible to view forces in a deterministic way. As additional information that might form a separate topic of discussion is that it was Einstein theory of relativity that shook the premise on which earlier laws were based. The prime focus of this theory is the fact that nothing is this world can be predicted with assurance (Uncertainty principle: shttp://en.wikipedia.org/wiki/Uncertainty_principle); in other words everything in this world is probabilistic.
A quick question that might arise in anyone's mind reading this article is if probability deals with uncertainty what utility it has in the real world. I will try to answer this question by quoting a statement by most Profs- no choice in this world is best; rather the best solution to anything is one that is optimal. This optimal solution can be found by optimizing a criteria which can be some cost function or complex equation. Hence probability is about finding an optimal solution to a problem and this assumption needs to be taken into account before using it. An example of cost function in case of a classifier that decides whether to cross road is seeking for minimum number of cars crossing the street.
b. Tools of probability: This part might seem a bit mathematical, however the intent is to provide details in KISS (Keep it Simple Stupid) framework.
Random Variable: It seems the mathematician were not quite happy with the use of the notation where probabilities are defined directly on sample space like P(rain occurring/clouds seen) . Hence for the purpose of simplifying notations, they introduced the concept of random variables that maps the sample space to a real line. Although most of the current literature provides a special definition of random variable (as written), I personally don’t understand the need to give any special distinction to RV. This can be party accounted to the reason that a random variable describes a random experiment where each outcome is associated with a certain probability that is defined by a probability density/mass function. It is also easy to understand a random variable by noting its name that says it is a variable (that can take on different values) that takes on values randomly.
A RV can be used to represent any experiment/process/phenomena in real life, say the reading of a machine can be assumed a RV that takes on multiple values. And here comes the idea of inference or optimal decision- is it possible to predict the state of machine based on the observables/reading.
Product and sum rule of probability: I read this statement in a famous book on PRML by Christopher Bishop that the entire field of probability is confined into the two rules- product and sum of probability. In other words, any operation in probability can be broken down into these two rules. Hence it is vital to recognize their importance.
Independent variables: The formulation of independent RVs is more of a tool than a concept that simplifies complex calculations and at the same time allows one to take into account information from the real world. For instance in case of two machines working independently, it is reasonable to assume that the two random variables representing their readings are independent. Two variable are said to be independent when it is not possible to make any(again any) prediction about a RV variable while other is known.
An important concept underlying the last idea is to understand why pair-wise independence doesn't imply mutual independence. Suppose there are two independent experiments where coins are being flipped. Let us name these RVs as X and Y and suppose there is another RV constructed called W that is defined as W=XY. Clearly W is independent of both X and Y individually since knowing one doesn't yield any information about W. However W is not independent of both at the same time since if we know both X and Y, W can be predicted.
c. Joint Random Variable: These are an extension of distribution of single random variables and used to represent joint experiments. In nutshell a joint experiment means to record the value of more than one RV simultaneously. Suppose Y=X+W, here f(X,Y) is a joint distribution function that can be used to estimate the probability of x and y occurring together. The only relevant information that is known to be extracted from JRM is joint moments like E[XY], that is also called correlation. One can also obtain central moments like E[(X-ux)(Y-uy)] called covariance. An interesting question that might occur to anyone is how are they important and what information they yield about the random variable.
Suppose two RV are independent then there is covariance is known to be zero. However covariance being zero can never imply two RVs being independent. This is due to the fact that correlation or covariance only yield information about the linearity between two RVs.
d. Moments of RV- Different moments of random variables called expectation yield some intuitive properties about the RV. For instance the first moment of a RV representing a worker's salary gives an average measure. Similarly the second measure provides an measure of variations in the RV.
The keys concepts discussed above should be sufficient to get hold of the basics of probability theory. I am sure with time I will have more to write on this topic. Thanks for reading it.
- Karan
Sunday, October 3, 2010
Grad School- UCSD
Before commencing this post I would like to mention that I am writing this post at 3:30 in the morning (or virtually night) lying on my stomach on my bed. I feel ashamed at not being able to formalize the idea of starting this blog again. This idea came to me while I was wasting away time for the last four months in the light of the fact that I will surely have to work.
Earlier readers- if any- would have also noticed that I have renamed the blog to quest to find answers and yes it is turning out to be something like that. Moreover I seldom feel that I am incapable of objectively answering these questions, however there always this positive side that drives away any fear. I have a lot to discuss in the coming posts esp about the following that covers a great deal of subjects
1. Object Recognition- :)
2. Crowdsourcing
3. Bhagwadgita
5. Questions
See you again
Karan
Earlier readers- if any- would have also noticed that I have renamed the blog to quest to find answers and yes it is turning out to be something like that. Moreover I seldom feel that I am incapable of objectively answering these questions, however there always this positive side that drives away any fear. I have a lot to discuss in the coming posts esp about the following that covers a great deal of subjects
1. Object Recognition- :)
2. Crowdsourcing
3. Bhagwadgita
5. Questions
See you again
Karan
Subscribe to:
Posts (Atom)