read a big log file backwards in python
need last errors first. file is like 3gb. readlines runs out of memory
Your discussion
This browser holds the management token for this post.
Save management token
Keep this private. Anyone with the token can resolve or delete this discussion.
4 replies
bump
bump again. anyone?
Not python but tac big.log | grep ERROR | head from the shell does what you want and stops early. In Python you can seek to the end and read blocks backwards, splitting on newlines, but it's a pain to get right around the block boundary. I'd pipe tac into the script and read stdin.
oh that is way easier. thanks, had given up on this