Warning Please do not post any new comments on this page.
This is a discussion archive first created in , although the comments contained were likely posted before and after this date.
See current discussion or the archives index.

Admin edit to MediaWiki page

Please change pages {{{from}}}-{{{to}}} to pages {{{from}}}–{{{to}}} at MediaWiki:Proofreadpage header template. —Justin (koavf)TCM 20:45, 1 November 2020 (UTC)

  Done: diff. Inductiveloadtalk/contribs 21:19, 1 November 2020 (UTC)
Nice. —Justin (koavf)TCM 21:22, 1 November 2020 (UTC)

16:09, 2 November 2020 (UTC)

Dotted line

Hi. Is there any template that simply can draw a dotted line? --Yousef (talk) 07:59, 3 November 2020 (UTC)

Try {{***}} (change the character as needed) for one on a separate line. Use {{...}} for an in-line line. Beeswaxcandle (talk) 08:07, 3 November 2020 (UTC)

Country Diary

I've started to copy over copyright-expired Country Diary entries by Thomas Coward from The Guardian - there are over 230 of them to do. For example:

How could their headers be improved? How about linking between them ("next" and "previous")? Andy Mabbett (Pigsonthewing); Talk to Andy; Andy's edits 12:06, 4 November 2020 (UTC)

@Pigsonthewing: I suggest:
Inductiveloadtalk/contribs 12:16, 4 November 2020 (UTC)
@Inductiveload: Thank you. I went for the former for now, as we currently have no other content from the relevant issues. I would still like to improve what is in the headers (currently "The Guardian by Thomas Coward"). Andy Mabbett (Pigsonthewing); Talk to Andy; Andy's edits 14:03, 4 November 2020 (UTC)
@Pigsonthewing: Sorry, that wasn't very clear! The two points were orthogonal. I meant "alternately" to using "next/previous" fields. It's kind of moot since the chances of adding more content from the exact issues is, shall we say, slim.
Other daily newspaper headers use something like [[../../../../]], [[../|6th July, 1920]] for the header title field, and then "Country Diary" for the section field. Inductiveloadtalk/contribs 14:11, 4 November 2020 (UTC)

Wiki of functions naming contest - Round 2

22:11, 5 November 2020 (UTC)

Need help for updating proofreadpage update on Telugu Wikisource

I initiated an update for Proofreadpage to try pagelist widget. In the process the index page edits are not showing the already populated entries. As part of this process I have deleted "MediaWiki:Proofreadpage index attributes". While trying to delete "MediaWiki:Proofreadpage_js_attributes", I created a blank page. Several years back, I think user:billinghurst helped setup proofreadpage on Telugu wikisource. This site uses MediaWiki:Proofreadpage_index instead of MediaWiki:Proofreadpage_index_template as mentioned on Proofread extension page. Pagelist widget error on Telugu wikisource bug can be seen for related info. Request user:billinghurst and others to help make the index page editing work again.--Arjunaraoc (talk) 06:35, 8 November 2020 (UTC)

@Arjunaraoc: te:MediaWiki:Proofreadpage_js_attributes was created by you today, I have deleted, as you should be using te:MediaWiki:Proofreadpage index data config for your config. — billinghurst sDrewth 06:49, 8 November 2020 (UTC)
@Arjunaraoc: have you stepped through mul:Wikisource:ProofreadPage/Improve index pages it is what I need to do as I so rarely play in that space. — billinghurst sDrewth 06:54, 8 November 2020 (UTC)
And if you are using the template fill gadget, the customisation is at MediaWiki:Gadget-Fill Index.js. Re the config change for Pagelist, you can see user:Inductiveload's edit for us at special:diff/10443824, and I am not really around the rest of that configuration change. — billinghurst sDrewth 07:02, 8 November 2020 (UTC)
Thanks billinghurst for your quick feedback and pointers. I got Proofreadpage and pagelist working. --Arjunaraoc (talk) 10:25, 8 November 2020 (UTC)

15:50, 9 November 2020 (UTC)

Template help

I looked in Category:Content templates and Category:Article templates (which seem like they should be merged, a separate point) and could not find a template for the top just basically saying "This article needs someone to come fix the endnotes/footnotes/whateveryoucallthem". If anyone knows it and wants to add it to my latest additions, much thanks. Wikisorce (talk) 04:53, 6 November 2020 (UTC)

@Wikisorce: Are you looking for Help:Footnotes and endnotes. Use <ref></ref>billinghurst sDrewth 07:19, 6 November 2020 (UTC)
i think they want a w:Template:Unreferenced, w:Template:More_citations_needed. but we do not really tag spam here like english wikipedia; we tend to link to pages to fix and people fix them. and we do not really merge categories either.
they seem to be scrapping and text dumping google books, without uploading to commons from IA. maybe some coaching about the scan backed process would be helpful. Slowking4Rama's revenge 23:36, 8 November 2020 (UTC)

@Slowking4: Thanks it must have been late at night I missed that componentry. @Wikisorce: We typically don't just scrape text, it is just a rubbish way to do book transcriptions. It was tried in the early days, and it was discarded as that crap just stayed as crap, and was too hard to fix. We rely on image files at Commons as a means to get reputable transcriptions, that can be further validated. — billinghurst sDrewth 23:56, 9 November 2020 (UTC)

three of the five works by Lysander Spooner, are at IA, and i linked to them at the page header. i leave the upload at commons, to the student. our new editor appears not to have returned. Slowking4Rama's revenge 22:10, 11 November 2020 (UTC)

Is this document worthy of uploading to Wikisource?

I've translated this scientific article into Russian, and I thought that maybe the English version could be uploaded to English Wikisource and my translation to Russian Wikisource. The original text is under the Creative Commons license. I used a couple of images from the article in Wikipedia, and I'm thinking of uploading the translation and providing a link from the Russian Wikipedia article directly to the translation in Russian Wikisource. Is this text worthy of inclusion into Wikisource? --CopperKettle (talk) 15:55, 12 November 2020 (UTC)

I think this article is within our scope and imo it would be great if you added it here.
However, I would like to point out a different problem I have noticed. You say that the article uses some images "from Wikipedia". These images probably come from the article Cerebral folate deficiency. The images in this article do not come from Wikipedia, if you click them, you find out that they are from Commons, a Wikipedia’s sister project, where they were uploaded under specific licenses saying which conditions have to be followed if somebody wants to use the pictures. For example the licence of Commons:File:Folic acid metabolism and 5-MTHF transport across the choroid plexus epithelium in the brain.png, says that among others "You must give appropriate credit" and "provide a link to the license", which was not done in the linked article. However, it is probably not a big problem, as the authors of the pictures are Sarah Mafi and Pierre-Antoine Faye, who are also co-authors of the article. Nevertheless, when a picture from Commons (or "from Wikipedia") is reused, generally the license conditions should be fulfilled. --Jan Kameníček (talk) 16:14, 12 November 2020 (UTC)
Thank you! It was me who uploaded the images from the paper by Mafi et al. to the Commons! --CopperKettle (talk) 17:35, 12 November 2020 (UTC)
Ah, I thought that the paper used the images from Commons and it is the other way. Then it is obviously OK, I am sorry for the confusing contribution. --Jan Kameníček (talk) 17:57, 12 November 2020 (UTC)

Another question

  • Actually, this work (Brain Sciences) raises an interesting question for dealing with works of this type. The issue the above articles comes from (vol. 10, no. 11) has 92 articles, more than many of the early volumes of this journal had in total. The articles themselves would not be difficult (assuming there were enough people able and willing) to add, but the actual issue page (Brain Sciences/Volume 10/Issue 11) would be very full, and probably difficult to search through. I have laid out how I had planned to list all of the articles on earlier sub-pages, but I feel like this method is very weak at a large scale. Any thoughts? TE(æ)A,ea. (talk) 19:09, 12 November 2020 (UTC).

OCR Gadget news

Hi all. Over at Phabricator there's been some progress made on fixing the OCR Gadget.

This function has been unreliable and for some works wholly inoperable since at least mid-2019. Over the last week or so there have been progress made on getting it fixed, and Inductiveload has updated our local Gadget (MediaWiki:Gadget-ocr.js). The long and short of which is that the   button in the toolbar should now be functional again.

There have been some unavoidable changes to surrounding infrastructure that may affect the details of the OCR text that is generated, but thanks to extremely thorough testing by Jan.Kamenicek these have hopefully been minimised.

Also, since part of the problem was due to corrupted data in the tool's cache, and this data had to be removed, works affected by this that returned OCR results really fast before, will now take significantly longer the first few pages. The first time a page from a given work is requested the tool schedules a background job to run OCR on the whole work and cache the results. Once that is complete any request for OCR returns the text from the cache immediately (fast), but in the mean time all the OCR processing must be done between the time you clock the   button and you getting the result back (slow). For most works this processing should complete within a couple of hours.

Finally, since the tool has been unreliable for a long time now, and there may have been multiple different causes of the failures people experienced, it would be very helpful if there was widespread testing; both on works that you know failed before, and on other works. There is some attention on this tool just now so good chances of getting any problems fixed, which may not be the case later on (it's a big complex codebase so it takes quite a bit of time to get familiar with it).

If you feel comfortable with that you can report problems directly in the Phabricator task linked above, but as Phabricator is a developer-oriented tool you can also simply report them here (or on my talk page if you prefer) and I'll make sure to summarise them there.

@Ineuw, Beleg Tâl: I believe you both at various times indicated interest in this issue, so pinging for "FYI" purposes.

PS. Also, please keep in mind that we have an alternate "Google OCR" gadget that you can turn on in your preferences, and each tool has its own strengths and weaknesses. I have both turned on and use whichever gives me best results on a given page. The Google OCR gadget has been very reliable in my experience, and often gives even better results. --Xover (talk) 08:40, 13 November 2020 (UTC)

Thanks for the reminder. — Ineuw (talk) 13:04, 13 November 2020 (UTC)
Well done!Mpaa (talk) 18:30, 13 November 2020 (UTC)

Systemic issue with page-scans.

Can someone check if this is just me, Index:Ruffhead - The Statutes at Large - vol 2.djvu or is there a quality problem with the scanned pages I've marked (!), it's NOT present on the original scans at IA, so one solution is for someone to re-generate the file. ShakespeareFan00 (talk) 15:47, 14 November 2020 (UTC)

@Hrishikes:,@Xover: Quality issues with Djvu are generally you area of expertise? ShakespeareFan00 (talk) 15:49, 14 November 2020 (UTC)

@ShakespeareFan00: Please describe the issue you're seeing. At a cursory glance, the page images here and at IA are entirely comparable. --Xover (talk) 16:53, 14 November 2020 (UTC)
The issue I am seeing is that on the pages I've marked, the side-notes are very blurred to the point of illegibility, The ones on IA when viewed directly are not. Whilst some compression loss is expected in djvu, I was not expecting illegible text in places.ShakespeareFan00 (talk) 19:15, 14 November 2020 (UTC)
@ShakespeareFan00: Ah, I see. This is indeed a problem with the (IA-generated) DjVu file. IA have used settings for background separation and compression that produces really poor results on this text. It's a typical "pathological case": the scan images are slightly too low-resolution for the text to begin with, compounded by the sidenotes being in very small type, and having very low contrast with the background, and then IA's DjVu software (LizardTech's encoder AIUI) has tried to separate the foreground (the text) from the background (the page paper)—which is an imperfect process—and then compress both separately with a lossy compression akin to JPEG in these respects. The end result is that they have produced a ~57MB DjVu from ~650MB of scan images (that's a better than 10:1 compression ratio on already compressed data!). If file size is your main concern then that's an incredibly good result, but for image quality and legibility not so much.
I can try to regenerate this from the source scans, but while that will almost certainly give better image quality (at the expense of file size), I can't guarantee that the OCR quality will be the same. The scan itself is really suboptimal on several scales that frustrate optimal OCR.
@Inductiveload: In case you don't have any to hand, here's a perfect example of the pathological case where IA's encoding fails. Zoom in on the sidenotes on this page (DjVu index 765, logical page 715). It looks like they've been written with a partially-dry magic marker, rather then printed or drawn with a pen. You can see the source at IA, showing that in addition to everything else the scan is slightly out of focus. This is why I'm deliberately not optimising for file size in my tools. --Xover (talk) 08:36, 15 November 2020 (UTC)
@Xover: yep, this is an example of a Mixed-Raster Content (MRC) encoder failing to segment the text correctly - it's basically treating that text like the background instead of as foreground. It's a shame the IA sets their (3rd party proprietary) encoder so aggressively, a "mere" 8:1 compression over JP2 would probably be a lot better. But I don't pay for their millions of hard drives, so I can't complain too much! They generally do a better job of compressing than the Big G. As you say, file size isn't critically important to us, but it does sometimes cause some pain when trying to sling files around.
If it's really necessary for the file to be regenerated and the new OCR isn't up to scratch, and the IA OCR is better and the OCR and Google OCR gadgets both do not work well, you could consider using the IA's own OCR and patching it over the new DjVu. Inductiveloadtalk/contribs 13:36, 15 November 2020 (UTC)
@ShakespeareFan00: I've generated and uploaded a new version without the aggressive compression (as you can see it's over 10x larger file size). The images should be comparable to what you see on IA now, but keep in mind you'll see a lot of cached thumbnails from the old version for a while. --Xover (talk) 14:38, 15 November 2020 (UTC)
@Xover: FYI, a compression at the c44 step (-size parameter) aiming for roughly 250kB image produces "OK" image quality with a legible page 765 (or at least as legible as the original), for a total DjVu size of about 200MB. I only add that parameter myself if it seem I can get away with it and without it it's producing large files compared to the output quality - I think you can save some of the file size by allowing it to throw away JP2 compression noise. In this case, aiming for 100MB file size produced pretty nasty artifacts, but 200MB and up was OK. I don't sweat it too much either way because WMF has lots of hard drives on one hand and the IA has the original JP2s on the other. Not that there's anything wrong with not caring about file size, but there's certainly a spectrum from "compressed to total unreadability" to "multi-GB files slavishly reproducing imperceptible high-frequency image data". Inductiveloadtalk/contribs 21:50, 15 November 2020 (UTC)

Two column layouts

What solutions are there at Wikisource for two column layouts, such as parallel text editions, with both texts presented, either on the same page, or opposite pages? Obviously, on a wiki, the content needs to be presented left-right in two columns rather than on different Wiki pages, and for ebook export, there is a second challenge that on any given ebook page rendering, the text needs to be roughly the same. As langauges tend to present in different but fairly consistent proportions, the text would need to "break" at certain points.

Is the only practical solution to this to use tables with columns / rable rows? or does Wikisource have other ways around this? JimKillock (talk) 11:33, 15 November 2020 (UTC)

I found the Help:Templates#Column_formatting section but if there is anymore to be said let me know, thank you/ JimKillock (talk) 12:47, 15 November 2020 (UTC)
If the bilingual text is placed on opposite pages, you may try the template {{Bilingual}}, see e. g. A Book of Czech Verse/K. J. Erben or Modern Czech Poetry/Cromwell at the corpse of Charles I.. --Jan Kameníček (talk) 22:29, 15 November 2020 (UTC)
Just forgot to ping @JimKillock:. --Jan Kameníček (talk) 22:32, 15 November 2020 (UTC)

15:37, 16 November 2020 (UTC)

Dynamic layouts in other than Main: space...

  1. Well someone enabled it. However, there are a number of templates (notably {{left sidenote}} and {{right sidenote}} that define specific classes and rules for ns104 (Page: namespace) Does someone have a list of these so that they or Mediawiki:Pagenumber.js can be updated accordingly? ( I also strong suspicions that some templates also make sweeping assumptions as to the initial page setup they expect and so aren't necessarily as compatible with dynamic layout as would be desirable.
  2. Another consideration is that the current dynamic layouts parameter sets in Mediawiki:Pagenumber.js do not yet have the additional rules to make it feasible to get an "approximation" of the relevant layouts in Page: namespace ( complicated by the side to side display of the page scans.). Mediawiki output in non Main namespace, and crucially that is not coming via a page tag, also lacks the column and region container elements that some of the layouts rely on.
  3. I am also needing some assistance in figuring out how to create my own 'layout' for test purposes, because the approach given in Help:Layout didn't actually work. Despite defining a custom layout per the documentation, it failed to show up as an option when looking through the available ones on a page.
  4. Also, currently whilst dynamic layouts seem to be active in Page: namespace, they don't seem to be fully active when editing or previewing in that namespace, which means that it's not as easy to check if a desired transcription will render correctly. Perhaps this is something that's being worked on?

ShakespeareFan00 (talk) 17:48, 17 November 2020 (UTC)

The instructions here no longer work. I have

mw.loader.load('//en.wikisource.org/w/index.php?title=MediaWiki:TemplateScript/proofreading.js&action=raw&ctype=text/javascript');

In my common.js, but sometimes the relevant entries fail to appear on my sidebar. What changed in terms of the sidebar recently that means the load method here no longer works as designed?

Alternatively can someone provide me with a list of the EXACT setting to use in preferences so that things actually work as the documentation claims? Thanks. ShakespeareFan00 (talk) 08:58, 18 November 2020 (UTC)

This works for me. I get the following entries in the sidebar with this line in my JS:
Page tools ⛭
    Add header
    Add footer
    Clean up OCR
    Make reference
    Convert to small-caps
    Convert to uppercase
Perhaps you need to flush your cache? I have found Firefox recently to be very reluctant to flush the JS cache without opening Developer Tools (F12) and setting it to "Disable HTTP Cache (when toolbox is open)". Inductiveloadtalk/contribs 09:21, 18 November 2020 (UTC)

Flawed template logic...

How are you supposed to check for extended usage of a templates additional parameters?

I added a line to {{Outside L}} and {{Outside R}} to do this, but it seems to be acting randomly, because it's putting into the relevant category, items I have checked do not use the additional parameters concerned.

Is the logic I've used flawed in some way, because I was trying to figure out if the code in these templates can be made simpler using templatestyles? Checking for actual usage is part of this. ShakespeareFan00 (talk) 15:48, 18 November 2020 (UTC)

Call for insights on ways to better communicate the work of the movement

ELappen (WMF) (talk) 18:56, 18 November 2020 (UTC)

First proposal reserved for OCR tool

The Wishlist survey just got opened for proposals, so I wanted to make sure the first one that was received was a request for a new OCR tool. Please comment there to register your thoughts. –MJLTalk 18:05, 16 November 2020 (UTC)

My biggest wish is that the Community Tech team as such gets fixed above all. They allow us to have wishes and vote about them, but they are not able to fulfill them, some of those chosen they even simply ignore (see the fifth one from the last year’s results) and after a year passes they start a new round of our vain hopes. The contributors to Wikimedia projects definitely deserve much better technical support than the annual fuss called Community Wishlist :-( --Jan Kameníček (talk) 20:04, 16 November 2020 (UTC)
@MJL, @Jan.Kamenicek: So far as I know, Community Tech are in the early phases of implementing the new OCR tool from last year's Community Wishlist and there is no need to resubmit that wish (Kaldari, do you happen to have up to date status on this?).
The problem is that they're a small team and tasked with completing all the accepted wishlist items from all the projects, some of which are rather large and complex, and some of which have dependencies on changes to other parts of MediaWiki that are owned by teams that are already up to their necks in other tasks. There are legitimate complaints to be made about the amount of resources allocated to the smaller projects, but those are properly addressed to the WMF leadership and the Board, and not individual developers doing the best they can within the limits they operate under.
But that being said, we can certainly do a lot better on our end, in specifying our wishes better, and coordinating our priorities across the different language Wikisourcen. For example, there were at least three different requests for OCR tools made last year, and it took a lot of discussion to determine whether they were similar enough to be merged. The Wikisource communities should put some effort into coordinating among ourselves before we complain too loudly. --Xover (talk) 21:39, 16 November 2020 (UTC)
I do believe that individual people of the team do their best, but the team as such does not work well, their small number being probably one of the causes. BTW there was only 1 wish from Wiktionary that made it among the five "winners" and after a year of vain waiting for fulfilling the promise they were told that they can just apply again (with zero chance as Wikipedia wishes will probably roll over smaller projects this year).
As for our end: I wrote the wish for OCR tool last year and I think I did my best to specify it as well as I could, consulting it with other people too (including you) and after discussing it with authors of the other two proposals I merged them into this one. I know it is not that much in comparison with the amount of work needed to create the tool, but contributors’ main task is to contribute and tech team’s task is to provide technical support for them. While the content of Wikimedia projects grows enormously every year, the technical backup stays far far behind.
You are right that my sighs here do not help anything, but I do not know of any systematic gathering of contributor’s opinions on tech team’s work by WMF. Not a long time ago I was asked to fulfill some questionnaire and was prepared to tell WMF everything, but it was full of questions about friendliness and harrassment in Wikiprojects, and practically nothing about our satisfaction with technical support. They do not seem interested in it, otherwise they would create some forum for people to speak about these problems. Finally I managed to find some space in the questionnaire to say what I was not asked about (that the Tech Team is drowning in a large amount of work and that phabricator is flooded with unsolved problems) but I am convinced I did not tell them anything they had not known, as everybody knows it. So please forgive me that I just vented my frustration. --Jan Kameníček (talk) 22:36, 16 November 2020 (UTC)
@Jan.Kamenicek: I could write a dissertation on the apparent disconnect between the WMF (as an organisation and at the leadership level) and the actual needs of the projects, most prominently exhibited in the lack of priority given to technical areas and especially for the smaller projects. But the thing is, I doubt you'd find much disagreement on that among the engineers working for the WMF; they just can't be too vocal about that in public given that they are employees (I know they advocate for more resources for technical areas internally though).
And please don't take my criticism of the community (failing to) hold up their end as criticism about your proposal in the 2020 Wishlist. What I meant was that, for example, right now the community should be discussing and refining our wishes for the 2022 Wishlist. We should be actively gathering input from all the different language Wikisourcen on what their priorities are, and then submit a unified and well specified package of wishes that represent the needs of all the Wikisourcen (who have much more in common than with the non-Wikisourcen). And we should do some active advocacy for our most critical wishes ahead of time. That is, we should be acting as a common community of interest, and not a bunch of uncoordinated Internet people. --Xover (talk) 07:51, 17 November 2020 (UTC)
expecting the volunteers to professionally curate their needs and lobby for them internally is a little "unrealistic". we had some progress stemming from the wikisource conference, but it required resources from WMF. WMF has historically not done UX well, leading to a dysfunctional "partnership". given the half-done visual editor roll out on wikisource, i do not expect much from wishlist, but by all means continue to try, it will keep the channel open. Slowking4Rama's revenge 13:23, 18 November 2020 (UTC)
@MJL, @Jan.Kamenicek, @Xover, @Slowking4: The Community Tech team is definitely still working on a new OCR tool for Wikisource. They are behind schedule though because the Watchlist Expiry project that they just finished took most of the year to complete (as it was more complicated than expected and the team was slowed down a bit by the pandemic, especially for folks with kids). Some of the preliminary investigation work for the new OCR tool has already been completed. (See T244100 and related subtasks.) They are also currently working on Ebook Export Improvement, which they've already made substantial progress on. The team is not going to ignore any of the top wishes. It may just take a while longer to address them all. Apologies for the delays! Kaldari (talk) 02:38, 19 November 2020 (UTC)
@Kaldari: Thanks very much for some good news. When I wrote that one of the top wishes was going to be ignored, I wrote about this edit. Fortunately it seems that the decision was revoked and the status of the wish has been changed for "pending" again. --Jan Kameníček (talk) 11:08, 19 November 2020 (UTC)

Community Wishlist Survey 2021

 

The 2021 Community Wishlist Survey is now open! This survey is the process where communities decide what the Community Tech team should work on over the next year. We encourage everyone to submit proposals until the deadline on 30 November, or comment on other proposals to help make them better. The communities will vote on the proposals between 8 December and 21 December.

The Community Tech team is focused on tools for experienced Wikimedia editors. You can write proposals in any language, and we will translate them for you. Thank you, and we look forward to seeing your proposals!

SGrabarczuk (WMF) 05:52, 20 November 2020 (UTC)

Hello, folks! My name is Ilana, and I'm the product manager for the Community Tech team. I would just like to clarify (in case there's any confusion) that the team will still work to address the Wikisource wishes from the 2020 survey. In other words, the 2020 Wikisource wishes will not be forgotten or neglected! In fact, we're already working on some wishes. The ebook export project is in progress, and we're begun research for the OCR project. For the other wishes, we'll also work on addressing them in the future as well. For this reason, there's no need to resubmit the 2020 wishes (from the top 5) in the 2021 survey. However, we look forward to reading and reviewing new Wikisource wishes in the 2021 survey. Thank you! --IFried (WMF) (talk) 00:36, 21 November 2020 (UTC)

Is there really no way?

Is there really no way to have a bot or shortcut link to automatically add archive.org pdfs as those proofreadable scans to wikisource with only a click or two? I'm looking at Author:Goldsworthy Lowes Dickinson which has links to the external source at Archive.org for each single work...yet none of them are on Wikisource and it seems like it would take hours to import all the source documents over and get them even just set up to be start being proofread. Wouldn't that be a tool worth creating to mass import such things? Peace.salam.shalom (talk) 05:08, 20 November 2020 (UTC)

@Peace.salam.shalom: There is such a tool: IA-Upload. It's still slightly fiddly, but combined with the "Fill Index" and "Import Pagelist" gadgets (enable in your settings), it's not too bad. I'm working on a way for users to provide me with a CSV file and I can run a batch job, but it's not there yet.
Some of the books have been uploaded to Commons already by Fæbot, for example File:Religion - a criticism and a forecast (IA religioncriticism00dick).pdf. With the two gadgets mentioned above, setting up an index isn't too miserable. Inductiveloadtalk/contribs 05:16, 20 November 2020 (UTC)
Oh, perhaps I am just confused then because there is not a link to see those scans (ie, Religion - a criticism and a forecast) on his userpage? There's not an automatic link [click here for the scans to be proofread] thing? Peace.salam.shalom (talk) 05:26, 20 November 2020 (UTC)
There is no bot to put links from enWS to IA. From an author page, feel that you can use {{ext scan link}} to add a link after the name of a work. No real value in a bot, as the detail at IA is never impeccable, never unique, full of variation. — billinghurst sDrewth 05:38, 20 November 2020 (UTC)
The scans were uploaded to Commons fairly recently by Fæbot, which doesn't create the Index pages for Wikisource. Formatting an Index page is virtually impossible to do using only the mediocre bibliographic data at the IA. Additionally, the page lists nearly always need some kind of manual intervention even in the best cases, and the scans need at least a cursory check for missing pages. With the two gadgets I mentioned above, most of the heavy lifting is done for you, but you still need to tidy it up manually in places.
Once an index page is created, you can change {{ext scan link}} to {{small scan link}}. Inductiveloadtalk/contribs 05:44, 20 November 2020 (UTC)
that being said, the cross project task of uploading works and transcribing them, and getting them found, is opaque and obtuse. it would be nice to have a query of works on IA and commons, that need an index, with a tool, or dashboard for an author page, so we could have a system, rather than rely solely on individual initiative. Slowking4Rama's revenge 02:08, 22 November 2020 (UTC)

Wikimedia meet

Just a note that one of the available tools for Wikimedians is Wikimedia meet. Not certain whether we have an uncalled need or not for some face-to-face sessions or not. I do wonder whether offering some occasional tutorial sessions could support new users. What are people's thoughts? — billinghurst sDrewth 02:07, 22 November 2020 (UTC)

it would be good to have a monthly call with meta:Wikisource Community User Group. then we could organize responses to strategy, and wishlist. User:VIGNERON (at IFLA) did a wikicite / wikisource on youtube. youtube.com/watch?v=fxyVlAHWz38 (disappointed by your filter of youtube again) some refresh of tutorials would be nice, also. Slowking4Rama's revenge 16:47, 22 November 2020 (UTC)

17:19, 23 November 2020 (UTC)

Bot-y Things?

I just noticed that not all the pages titled "Letter to..." are in Category:Letters and it seemed like the sort of thing a bot/automated task should be set up to not only do the backlog now, but keep up with in the future when somebody adds such a letter. Not all letters will be titled "Letter to...", but all works titled "Letter to..." or "Correspondence of/between..." can be safely assumed to belong in the category. But I have no idea where to find bot lists. Peace.salam.shalom (talk) 15:34, 24 November 2020 (UTC)

We would not typically put all subpages of a work into a category. We would usually do the top level of the work only for this example. If there a subpage of the work that was different from the parent, then we may do it. — billinghurst sDrewth 01:22, 26 November 2020 (UTC)

Page preloader gadget

 

There is a new gadget for pre-loading the next scan page. It's mostly cribbed from mulWS, but it tweaked so it works in non-edit mode. This speeds up the loading of the "next" link when in the Page namespace. When preloaded, a green underbar is shown on the next page link (see right).

It can be found in the "Editing tools for Page: namespace" section of Special:Preferences#mw-prefsection-gadgets.

Inductiveloadtalk/contribs 11:11, 18 November 2020 (UTC)

This gadget may or may not be needed once this patch lands (hopefully next week): phab:T230689. Inductiveloadtalk/contribs 14:47, 28 November 2020 (UTC)

Gadgetisation of the PageNumbers/Dynamic Layouts code: step 1

The first step of unpicking the dynamic layout and page numbers Gordian knot is to shunt all the code from Mediawiki:Common.js to gadget-based code. This is because Common.js code can't be easily turned off to allow a local or sandbox version to be used for development.

I have now moved the code to a gadget, but you need to take some steps to disable the Commons.js code as well. If you would like to test it:

  • Visit Special:ApiSandbox#action=options&format=json&change=userjs-disable-nongadget-pagenumbers=1 and submit the query. You will be prompted to automatically add a CSRF token, just click "submit".
  • This sets the userjs-disable-nongadget-pagenumbers user option to 1, which disables the current code in Mediawiki:Common.js
  • Page numbers and dynamic layouts should now be off.
  • Visit your gadget prefs and enable the "Display Options" gadget under the "Development" section.
  • You should now see the page numbers and dynamic layouts again. There should be no visible difference between the gadget version and the current version.
  • If you want to go back, disable the gadget and use the API sandbox in the same way, but set userjs-disable-nongadget-pagenumbers to 0.

If anything is not working, please report it here.

If no-one reports any issues in a week or so, we can make the gadget default-enabled, disable the code in common.js and move forward with the process of uncrustifying the code. Inductiveloadtalk/contribs 14:38, 18 November 2020 (UTC)

@Inductiveload: Works for me. BTW I left some feedback on your user talk page, concerning a situation that arose whilst testing something.ShakespeareFan00 (talk)
The code is now loaded as a default gadget and the Mediawiki:Common.js code is removed:
This means that it is now possible to disable the pagenumbers/layouts code easy and this makes it easier for maintainers to test other versions before going live.
It may take a short while for the changes to propagate through caches. As before, if breakage occurs, please let me know in this thread. Inductiveloadtalk/contribs 13:41, 28 November 2020 (UTC)

Using Wikisource tech on Wikibooks and elsewhere: MediaWiki extension

Hi all, I recently asked on en.Wikibooks whether the project could support ebook development better by bringing across the layout features that Wikisource employs, such as layout templates, navigation between chapters, ebook export tools. Wikibooks admins however felt that this would be difficult to deploy because of decisions they had made about their own set up. In the discussion, it was suggested that the better way to do this might be to develop a MediaWiki extension that contained the core features and made it easier to deploy. Since the people here know the tech, I thought I would ask if this seems sensible. Is a MediaWiki extension the right way to share Wikisource's layout tech? is this something this project would help with? JimKillock (talk) 04:12, 12 November 2020 (UTC)

@JimKillock: Our toggled layouts are independent of our text, and grabbing layouts is just a matter of looking what is in our mediawiki:common.js; our texts layer then just flow.

With regard to WSexport, I know that it is an independent tool that is having some rewrite. If you think that it should be plying its trade to more wikis, then it would be a matter of a phabricator ticket to get them to make those changes. — billinghurst sDrewth 08:47, 12 November 2020 (UTC)

Thanks, I have a basic knowledge of html and css, but Wikimedia / MediaWiki's technical workings are new to me. These are the items I am looking to port across
  • optional narrow single column formats;
  • optional serif fonts for body text
  • Differing styles for headings and so on (not always underlines)
  • easy navigation bars (last page, next page)
  • optional export links to common ebook formats
  • Left and right marginalia
So two questionsn regarding those:
  • Are all of those fairly simple CSS / JS fixes?
  • Are the engines for each of these just a Template to copy across otherwise?
And on WSExport,
@JimKillock: there are a few parts to this.
At enWS, the dynamic layouts are provided by Javascript which forces certain content to have the switchable styles. This code is currently a mess and split among various JS and CSS files. It is planned to transition it to a proper gadget. Not all of it will be needed for Wikibooks as there is special code for the page numbers/links that's mixed up with it. It will probably be easier for Wikibooks to make their own gadget for this, as it just needs to add some CSS and then set a relevant on the main content div.
Left and right marginalia are tricky and even enWS has not hit on a perfect solution yet.
Navigation bars are part of out templates (like {{header}}) and Wikibooks can do something similar. It's just a normal wikilink. {{header}} is probably not suitable for a direct port to Wikibooks and it's very custom-built for Wikisource, but you could use some concepts.
The WSexport tool is provided by an external service that runs on the Toolforge infrastructure. You can file bugs and issues at https://phabricator.wikimedia.org/tag/wikisource_export/. WSexport does not use the dynamic layout stuff, but it does apply special CSS from Mediawiki:Epub.css.
There is a very small gadget at enWS used to add the links for WSexport, but the rest of the logic is on the Toolforge server. Inductiveloadtalk/contribs 12:32, 12 November 2020 (UTC)
It's been pointed out to me that at first stage, the layout options are more for the editors to choose than users. The idea being that certain book formats could be chosen as a fixed decision by the editor / creator and applied by a simple template. Does that make sense? If so, I will start with that approach as it is quicker, easier and much more within my ability to action without asking for help. JimKillock (talk) 11:25, 15 November 2020 (UTC)
Ji @Inductiveload: could you point me to the relevant CSS files for the (narrow column, etc) layouts? That way I should be able to trial appying these via [[[mw:Help:TemplateStyles]]. Thank you! JimKillock (talk) 21:33, 17 November 2020 (UTC)
@JimKillock:: it's not quite as simple (yet) as a straight CSS file, but the styles (such as they are) are found at Mediawiki:PageNumbers.js in the self.ws_layouts object. Inductiveloadtalk/contribs 12:40, 18 November 2020 (UTC)
@Inductiveload: thank you, that is quite a small amount of CSS per layout, just a few lines it seems. I will take a look later. JimKillock (talk) 16:01, 18 November 2020 (UTC)
@Inductiveload: Ok, I got as far as creating a template to call the CSS, and adding a CSS file at wikibooks:la:Formula:NarrowColumn/Styles.css – however something is wrong that the css parser is not picking up: I can only save the file by commenting the whole thing out. If you can spot why I would be very grateful! And sorry for asking about something trivial JimKillock (talk) 21:20, 18 November 2020 (UTC)
Hi @JimKillock:, I don’t understand the tech, but I want to be able to copy books to WB where they could be annotated. I had help from Quite Unusual at Wikibooks to produce this copy of Economic Sophisms and he transferred(?) some templates over but there seems to be little work done before or since. Could your idea cover copying books from WS to WB? Please excuse by ignorance, I am primarily a proofreader here. Cheers Zoeannl (talk) 00:57, 18 November 2020 (UTC)
The changes I am after wouldn't automate copying across, but they would help with the book layouts once copied, in particular if you wanted your book to appear with a narrow centre column, and to export to ebook format. JimKillock (talk) 08:35, 18 November 2020 (UTC)
@Zoeannl: Looking at your book, it is the same missing templates that I am playing with on Wikibooks and have copied to la.wikibooks, eg x-larger and x-smaller etc. These are very easy to move, you just copy and paste them. The templates are at /wiki/Template:TemplateName eg Template:Larger for {{larger}} - you could easily copy these over yourself. If I implement the CSS for narrow columns, that can be applied and then your presentation issues are more or less solved. And ebook export at Wikisource is next on my list. JimKillock (talk) 16:01, 18 November 2020 (UTC)
@Zoeannl: I added some of the missing templates, and applied the work in progress version of the narrow column template, so you can see how east this is. JimKillock (talk) 18:21, 22 November 2020 (UTC)
@Zoeannl: I have done some more on the book, and it basically all seems to be working now. Can you let me know if anything more needs doing? JimKillock (talk) 20:16, 30 November 2020 (UTC)

Update: someway there

Hi all, I am someway towards migrating the relevant Wikisource features to Wikibooks, starting with Latin Wikibooks. I have two templates, to open and close the divs which make the column, and a style sheet attached to the first.

Questions:

  1. The styles don't seem to be working out the serif fonts yet. This seems to be due to the way the MediaWiki parser is working, it seems to be over-riding the font values. Any idea how I fix this?
  2. is there a better way to open and close the wrapper divs than using two separate templates, one to open them, and the other to close them?
  3. I have not understood the Wikisource template divs, I think, but just pushed the three across as per the stylesheet, you may have advice on what I should be doing here.

Thanks for any help. JimKillock (talk) 15:43, 21 November 2020 (UTC)

@Inductiveload: @Billinghurst: I am hoping one of you may be able to help with these three requests :) JimKillock (talk) 18:24, 22 November 2020 (UTC)
I am not really a complex css person, I am not help to you. — billinghurst sDrewth 01:25, 26 November 2020 (UTC)
Thanks, this is resolved. More stupid errors my side.
I am still unhappy that I am using two templates, one {{NarrowColumn}} to open the divs, the other {{NarrowColumnEnd}} to close them, with the page content between them. This feels like a great way for users to break things. Is there an alternative way to do this? JimKillock (talk) 18:26, 26 November 2020 (UTC)
OK, I think I have resolved this now. JimKillock (talk) 20:18, 26 November 2020 (UTC)

WSExport and Phabricator ticket

Hi billinghurst Ealier you remarked that If you think that it should be plying its trade to more wikis, then it would be a matter of a phabricator ticket to get them to make those changes — how do I go about that? This is hopefully the last thing I need to do to complete this now, apologies for the many queries. JimKillock (talk) 21:20, 26 November 2020 (UTC)

This is resolved! Thank you for the information earlier. JimKillock (talk) 21:24, 26 November 2020 (UTC)

Wikidata descriptions changes to be included more often in Recent Changes and Watchlist

17:45, 30 November 2020 (UTC)

Please see also c:Commons:Village pump/Copyright#The years of registration vs. publication of the United Nations Treaty Series since Volume 401. Perhaps we may host treaties registered in recent copyright-declared volumes only from different sources, like US governmental sources for American treaties. Volumes 1121, 1199, 1200, and 1237 were registered in 1979 to 1981 but published in 1996 or 1998 with copyright notices.--Jusjih (talk) 21:31, 30 November 2020 (UTC)

SpBot and section resolved

Does the SpBot take into account the template {{section resolved}} when archiving discussions from this page? --Jan Kameníček (talk) 14:55, 26 November 2020 (UTC)

I am asking because WS:Copyright discussions has a notice "SpBot archives all sections tagged with {{section resolved|1=~~~~}} after 7 days" at the top, while this page does not contain such a notice and some sections have not been archived yet although the template was placed there almost a month ago. I have placed the template to one of the sections recently so I would just like to know whether it is going to have any effect. --Jan Kameníček (talk) 07:54, 27 November 2020 (UTC)
@Jan.Kamenicek: The behaviour of SpBot is configured per-page with the template {{autoarchive resolved section}}. The two main parameters of interest here are age and timecompare. age defines the number of days after which SpBot considers a thread eligible for archiving. When making this assessment it refers to the value of the timecompare parameter: if it has the value resolved the bot determines the age of the thread from the timestamp in the {{section resolved}} template, and if it has any other value (including empty) it looks at the oldest timestamp in the thread (actually, I think that's a documentation error; I'm pretty sure it looks at the newest timestamp).
In practice this means threads on WS:CV are never archived, unless they contain {{section resolved}} in which case they are archived 7 days after the date in the template. Threads on WS:S are archived 31 days after the last timestamp in the thread (modulo the doc confusion mentioned above). So for the thread you were interested in, adding {{section resolved}} probably postponed archiving until 31 days after you added the template, rather than let it be archived 31 days after the last regular message in the thread. For this reason I usually use {{closed}} rather than (well, in addition to) {{section resolved}} for threads that have outlived their productive lifetime on WS:S. --Xover (talk) 10:31, 27 November 2020 (UTC)
@Xover: Thanks for explanation. Do I understand it right that the correct way of accelerating uncontroversial archivation is using {{closed}}? --Jan Kameníček (talk) 23:29, 27 November 2020 (UTC)
@Jan.Kamenicek: No, {{closed}} doesn't affect archiving either; but it signals that the discussion is over and no further input is desired. You can also use {{cot}} and {{cob}} to just collapse the section contents without the quasi-official connotation of {{closed}}. PS. Apologies for the tardy reply. --Xover (talk) 16:45, 1 December 2020 (UTC)
I see. However, what I originally wanted to achieve was speeding the archivation so that the discussion in the archives could be linked to because it does not make sense to link here as it is going to disappear from here after some time. So I will wait until the bot does its job. --Jan Kameníček (talk) 16:55, 1 December 2020 (UTC)
@Jan.Kamenicek: Ah. In that case, you could use a permanent link that points to the relevant section at a specific revision. It requires manually finding the revision id and constructing the link, which is really hard to figure out the first time and really tedious and fiddly every time you want to do it. I made myself a script to make it easier, but it's rather buggy in some places. If you want to try it out you can stuff mw.loader.load('//en.wikipedia.org/w/index.php?title=User:Xover/EasyLinks.js&action=raw&ctype=text/javascript'); into your common.js. No warranties, but it'll give you a link next to each section heading labelled "permalink", which when clicked puts a permanent link to that section on your clipboard. For example, here is a link to this section as it was right before I posted this message: SpBot and section resolved. You can also get non-permanent links to a section, and links to a diff (when viewing a diff). At some point I'll get around to polishing it up suitable for others to use, but it works well enough for my needs so I keep putting it off. :) --Xover (talk) 17:16, 1 December 2020 (UTC)
@Xover: Creating a link to the section at a specific revision is a very good idea. I think I found quite an easy way: I found and opened the revision, and then clicked on the title of the section in the Contents box, which generated the desired link in the browser’s adress bar ( https://en.wikisource.org/w/index.php?title=Wikisource:Scriptorium&oldid=10674984#Self-published_lectures ). Thanks very much for the advice. --Jan Kameníček (talk) 18:46, 1 December 2020 (UTC)

Moynihan report 1965

I thought I might find the report written by Daniel Patrick Moynihan, commonly referred to as the Moynihan report 1965, and more formally as The Negro Family: The Case For National Action. the report is discussed briefly in his biography, and in more detail in w:en:The_Negro_Family:_The_Case_For_National_Action. Both articles have links, although I just added a discussion to the respective talk pages noting that the original link is dead, and proposing to change it with a working link. Most of the material is captured in Internet archive links, although a list of references available at the original is missing from the Internet archive. I'm bringing this up here for two reasons:

  1. Given that the work is public domain, and a very important document, it would seem appropriate to include it here
  2. I'm particularly interested in the "numerous tables and graphs" mentioned on the bottom of the first page of the document which are not shown at the Department of Labor site. I think those would also be useful — I'm hoping someone here has more experience with how to track them down and add them.--Sphilbrick (talk) 14:36, 30 November 2020 (UTC)
the us DOL link has only a web1.0 text dump [17]. not scanned at IA [18] here is a google books copy [19] (you could download it and upload at IA and commons) - Slowking4Rama's revenge 15:14, 1 December 2020 (UTC)
thanks very much, the diagrams appear to be Dept of Labor, but might want to check, before transfer. the OCR does not like the two columns, so we may need to paste the Dept of Labor text in. Slowking4Rama's revenge 21:19, 7 December 2020 (UTC)
Thanks for doing this, very helpful.--Sphilbrick (talk) 20:49, 8 December 2020 (UTC)

IA Upload not working

The IA Upload tool has not worked for me for several days; it seems from its logs not to be working for many other people, if not everyone. I've raised a Phabricator ticket. Andy Mabbett (Pigsonthewing); Talk to Andy; Andy's edits 19:39, 23 November 2020 (UTC)

No response there; still not working, at my most recent attempt. Andy Mabbett (Pigsonthewing); Talk to Andy; Andy's edits 18:40, 11 December 2020 (UTC)
Search commons for the ia id, a pdf might be there.--RaboKarbakian (talk) 22:00, 11 December 2020 (UTC)
Same for me. A few days ago I still could get on https://ia-upload.toolforge.org/. But now I get a message "503 Service Not Available". --Dick Bos (talk) 08:30, 23 January 2021 (UTC)