A key question in numerosity cognition is whether numerosity processing is based on quantity of items, rather than other low-level descriptions such as contrast or density. We can shed light on this question by investigating the cortical remapping features of numerosity adaptation across saccades, compared with those of low-level properties. As can be seen from previous studies, when the observer was sweeping across a stimulus, there was a spatiotopic representation of the stimulus, forming an isomorphic map in the visual cortex anchored in stable real-world coordinates. The remapping was based on transformation of retinotopic mapping of the neurons that occurred in the primary visual cortex. This transformation, however, was proposed to exist in high-level descriptions selectively by means of integrating visual properties with eye-movement information across saccades. Therefore, distinct mechanisms can be revealed if the remappings show different characteristics between numerosity and low-level properties such as contrast or density, which is helpful in supporting the statement that numerosity processing is relatively independent of primary visual cues. Object-based remapping was thought to be possible when stimuli moved without saccades. The mechanism underlying this kind of mapping is not yet clearly demonstrated. Specifically, if this remapping is due to object-file updating, then all visual properties could be remapped when objects move without saccades, despite the different processing levels of them. By contrast, if the object-based remapping is based on a similar mechanism to that with saccades, then a distinction would appear in remapping features of descriptions in different levels: only higher-level descriptions’ aftereffects could be remapped to final positions when objects moved. Therefore, exploring remapping features when objects are moving without saccades is an effective way to study the mechanism of object-based remapping without saccades. In our study, remapping characteristics of numerosity adaptation aftereffects were investigated systematically when relative motion occurred between the target and the observing eyes, compared with those of contrast adaptation aftereffects. Two experiments were included. In Experiment 1, subjects were asked to shift their fixations to a new position after adapting to the stimulus. The adaptation effects in the same and different retinae/screen positions were compared. The numerosity and contrast adaptation were tested respectively. As a result, numerosity adaptation effects showed spatiotopic remapping features to some extent, whereas the remapping of contrast adaptation aftereffects appeared to be purely retinotopic. According to the result, we proposed that numerosity was a higher-level description of stimuli, which can be integrated with eye-moving information and built up across saccades. In Experiment 2, with a stable fixation and moving adapting targets, we determined the numerosity and contrast adaptation aftereffects in the original and final positions of targets, respectively. The results suggested that numerical adaptation aftereffects can be remapped to the final position of targets. On the contrary, contrast adaptation aftereffects could not be remapped to the final position of moved targets, showing a completely retinotopic remapping feature. The distinct results in Experiment 2 underlined further differences in numerosity and contrast processing, as well as an identical mechanism in remapping features of relatively moving objects with and without saccades. In conclusion, cortical remapping of numerosity adaptation aftereffects was shown to be not constrained to the retinotopic level in our study. With saccades, it was shown to be spatiotopic; without saccades, the numerical aftereffects could also be remapped in the final position of moving objects. On the contrary, contrast showed completely retinotopic remapping features. These results provide new evidence to support the view that numerosity is processed at higher-level rather than being completely dependent on inferring of low-level visual cues. Moreover, we propose that the information of relative motion can be integrated with higher-level properties of stimuli no matter there are saccades or not.