3. How can I check data for missing values?
For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/
2019-10-02: The community is currently in read-only mode

How can I check data for missing values?

  • A

    Hi all,

    I have bought some intraday data but I would use pandas to check if something is missing in case of 5-minutes bar.

    How can I check it and fill any missing data?

    0

  • 1. Determine if your timestamps are spaced equally (no gaps that are not equal to 5 minutes)

    Assuming that your timestamps are stored in the index column of the pandas dataframe df:

    timestamps = list(df.index.values)
delta_t = list(set(timestamps[1:])-set(timestamps[:-1]))

    And check (for in-market timestamps) whether or not the any element in delta_t is not equal to 5 minutes.

    2. Filling Values in Pandas

    Pandas has extensive data gap filling capabilities - which you can learn more about here. But for stock/forex data I have used the backfill method, which uses the previous value to fill in the gap. This results in "flats" where the data forms a horizontal line on the chart where data is missing.

    0
    A
     1 Reply
  • A

    @Brandon-Johnson thanks a lot. I will try

    0
Log in to reply
 
});