Dariush

Members
  • Content count

    32
  • Joined

  • Last visited

  1. IMO a good approach would be to do this: 1. Limit the permanent storage in the following way: first two months everything is stored, after the two months no more than, e.g., 5k items. Give an option to buy full permanent storage for selected feeds. 2. Don't throttle feeds with more than one user. 3. Actually detect which feeds aren't read - if, let's say, 50% of the items are never shown on screen when they hit the age of 1 month, the feed is considered dormant and can be throttled. 4. Have a regenerating quota per feed - for example, have it regenerate at a speed of 150/hour to a cap of 5k. Having this value low will cause the feed to be throttled. Basically, more soft caps, less hard ones. I understand that you cannot micromanage everything, but I don't think so many people are sending requests for manual limit lifting that it's unfeasible to handle them on a case-by-case basis, plus it's not like people with malicious feeds will be pinging the developers anyway. I only have one throttled feed, so pretty please?
  2. Yes, that's the case for me. Throttling the feeds that need the opposite, as I described in a nearby thread recently, with no warning, notification or even a changelog entry, was not a particularly user-friendly move, to be honest. If anything, the throttling should have been done for feeds that contain a lot of items at the moment of crawl, not cumulatively. …And I cannot even boost the feed, instead receiving the error "Boosting is disabled for this URL. The most common reason for this is a publisher restriction.". Could the throttling be lifted on this feed, please? I am a perfectionist and missing items really hurts my soul.
  3. Any updates about this?
  4. See for yourself: This feed is sufficiently high-volume that it immediately started hitting the update cap and thus losing items. My other similar (and in fact, smaller) feeds are still updating every 30 minutes. Also, I probably requested this already, but notifying the user about hitting the 100-item cap would be really, really appreciated. I've been lucky that I happened to check crawler history only a few days after this occured over an unrelated question.
  5. The feed in question is for a user content aggregator (as, I suspect, most of this kind of high-volume feeds are) with erratic upload patterns (Crawler history shows anywhere between 5 and 90 new items per crawl every 30 minutes), so I'm more worried that someone does a batch upload of 100+ posts and a part of them will not be crawled. And I'm not suggesting removing the protection (the presence of which is completely understandable), just relaxing it.
  6. I know about the variable crawl speed, I was trying to refer to the maximum unboosted speed of crawling every 30 minutes… except for some reason I thought there were 12 hours in a day. Oops. The actual maximum figure is actually 4800 per day, which is much better, but I'm still a bit uncomfortable with the possibility of missing items in case of large single feed updates. I'd still love to see an increased crawl size, since, as I explained in the previous post, it would affect only a tiny subset of feeds.
  7. So it's physically impossible to be fully updated on any feed generating more than 2400 items per day without boosting? That's pretty disheartening, to be honest - I can tweak my own feeds, but some of the external ones I'm subscribed to are dangerously close to that limit. Could this possibly be increased to 200, or at least 150 items per crawl? The vast majority of feeds only return like 20 latest items, so this change should only affect a tiny subset of customized ones, and most of _those_ only update a small subset of their items, so your systems should not be impacted in any significant way, but for people with actual high-volume feeds (like me) this would make life so much better. Also, notifying the user that a feed contains items that were not crawled (like a 'this feed has some problems' red bar on top, but not reappearing after being closed) would also not be amiss.
  8. Recently I started noticing items missing from some feeds generated and hosted on my computer and decided to investigate. I created a feed (http://dariush.duckdns.org/test3.xml) that consists purely of items numbered from 0 to 600, with random pubdates from this month. I added it to Inoreader and instead of seeing the 600 items I only got 100 latest ones. I rerolled the dates without changing the contents and got around 70 new items - only those that got a pubdate within the latest 100 got crawled. Is Inoreader really limited to 100 latest items or am I doing something wrong? I distinctly remember being able to pull more than that.
  9. That's… really off the mark. I specifically said it doesn't work _without_ r=o. The following two queries (API key omitted, obviously) have old timestamps two weeks apart (the first is from 08/06, the second is from 07/25), but return the same set of latest items, the newest of which as of right now has the timestamp 1502607600 (so yesterday) and thus should not have been retrieved. https://www.inoreader.com/reader/api/0/stream/contents/feed/http://feeds.feedburner.com/CrackedRSS?AppId=1000000562&AppKey=<>&ot=1502000000 https://www.inoreader.com/reader/api/0/stream/contents/feed/http://feeds.feedburner.com/CrackedRSS?AppId=1000000562&AppKey=<>&ot=1501000000
  10. Let's say I want to grab the feed https://www.inoreader.com/reader/api/0/stream/contents/feed/http://feeds.feedburner.com/CrackedRSS . If I pass in ot parameter any past timestamp, I get the same set of latest items. If I pass in any future timestamp, I get a blank result. It seems to work correctly with r=o, but I need the newest first order. Am I missing something or this is indeed bugged? Related: is it possible to get next page of results in some more convenient way than grabbing the timestamp of the last item and passing it in ot?
  11. I extensively customize Inoreader with CSS, and one of those customizations is highlighting items with variously coloured borders depending on keywords in title and article content. This works perfectly in expanded and list views, but in card view content isn't present, which limits me to title-based highlighting. It would be extremely useful to me if instead of loading on clicking a card the content was always present and invisible. Otherwise, I'll be forced to butcher the expanded view into a ghetto-card view for certain feed URLs, which would be just extremely unpleasant.
  12. Jump to earliest unread article

    I wrote a userscript to do this exact thing: $(document).keydown(function (e) { switch (e.which) { case 13: // enter var next = $(".article_unreaded")[0]; next.parentNode.scrollTop = next.offsetTop; next.click(); break; default: return; // exit this handler for other keys } }); This scrolls to the first unread article and clicks it to mark it as read, so we can scroll to the next one later.
  13. No, that's not it. As I said in the crossed-out part, other feeds from this server were working and the server isn't accessible 24/7. The crawler log showed this error for 24+ hours straight. The only URL is unchanged, so I assume you meant XML. Here is a minimized version trimmed to three entries - first one displays as normal in Firefox's preview, second one contains the backspace, which breaks this item and all the following ones; removing the backspace shows all three items normally and allows the feed to be read by Inoreader. Posting a plaintext version is pointless, since the character gets stripped along the way.
  14. I have several feeds hosted on my server. Of them, exactly one (http://dariush.duckdns.org/dailypixiv.xml) broke between 1 and 2 days ago with the above error. The feed is accessible from the internet (via FeedValidator). This server is accesible daily between around 9.30 and 23.00 GMT. Edit: by the universal internet law of figuring any problem out five minutes after asking for help with it, I found the culprit: this feed had a backspace character (looks like 'BS' in text editor) in it. Removing it allowed Inoreader to fetch the feed. However, the error is still definitely not correct - even if it breaks the feed, this shouldn't have anything to do with route to host. Irrelevant to Inoreader, but why does it break the feed itself? Firefox RSS preview only shows items up to the one containing this character and FeedValidator fails to validate the feed.
  15. Allow to set a flag on folder 'Sequential' that will do two things: 1) Force manual sort on the folder; 2) When the folder is viewed, instead of mixing articles from all feeds they will be displayed in order - first all articles from the first feed, then a separator, then the second feed and so on. The views of every individual feed will be used, as opposed to the folder imposing the same view on all feeds inside.