Experience (exp) is a modifyer used to represent the raise in skill a unit or squad gets when defeating an enemy. This "skill" can only be aquired by fireing at enemy units and a rather large bonus is added for the killing blow. Experience is exponential in the sence of scale - to get a smal bonus from experience, you don't have to do much, but if you wish to double the effect, you will have to work for that several times as much
The effect of having a skilled unit is usually insignificant, as the skill often comes at the cost of it's fallen brethren, leaving him at a disadvantage. Sometimes, the experience does come into play, as units will get a HitPoint boost (rarely going over 2x and averaging at 1.5x before death) and an accuracy boost.
Mortars, howitzers and mobile artillery do not get any experience ever (their experience is reset before fire, so you might see if you got a big hit or not with their previous round). This is due to artillery units having their own accuracy mechanic that experience interfiers with. For more information on accuracy, see Accuracy