Wikisource:Scriptorium/Archives/2019-10

Please do not post any new comments on this page.
This is a discussion archive first created in October 2019, although the comments contained were likely posted before and after this date.
See current discussion or the archives index.

Is Wikisource no longer getting indexed by Google and other search engines?

Several recently-created pages seem to be completely absent from Google. If I search for the complete title, and even add "site:wikisource.org", Google comes up with nothing. I see the same with Bing.

For example, I tried searching for "Ritual of the Order of the Eastern Star", which is currently at the top of the "New Texts" section of the front page, and again nothing (the index file comes up, but not the actual Wikisource text).

Has Wikisource done something to deter indexing, such as the use of a robots.txt file? If so, why? It makes Wikisource content harder to find, which is surely inimical to the success of the project. Grover cleveland (talk) 09:48, 4 October 2019 (UTC)

I've done some more digging, using the Wikisource:Works/2019 page. Notes on the Ornithology of Oxfordshire, Aplin 1899-1900 (added August 1st, so more than two months old) appears to be absent from Google. Other (older) pages that link to it are hit, though. It appears that Google isn't indexing newly-created pages, but is continuing to update its indices of older pages. Grover cleveland (talk) 10:12, 4 October 2019 (UTC)

Copyright and deletion discussions needing community input in October 2019

The following copyright discussions and proposed deletions discussions have been open for more than 14 days, and with more than 14 days since the last comments, without a clear consensus having emerged. This is typically (but not always) because the issue is not clear cut or revolves around either interpretation of policy, personal preference within the scope afforded by policy, or other judgement calls (possibly in the face of imperfect information). In order to resolve these discussions it would be valuable with wider input from the community.

Copyright discussions require some understanding of copyright and our copyright policy, but often the sticking points are not intricate questions of law so one need not be an intellectual property lawyer to provide valuable input (most actual copyright questions are clear cut, so it's usually not these that linger). For other discussions it is simply the low number of participants that makes determining a consensus challenging, and so any further input on the matter would be helpful. In some cases, even "I have no opinion on this matter" would be helpful in that it tells us that this is a question the community is comfortable letting the generally low number of participants in such discussions decide.

Copyright discussions

Proposed deletions

Note that while these are discussions that have lingered the longest without resolution, all discussions on these pages would benefit from wider input. Even if you just agree with everyone else on an obvious case, noting your agreement documents and makes obvious that fact in a way the absence of comments does not. The same reasoning applies for noting your dissent even if everyone else has voted otherwise: it is good to document that a decision was not unanimous.

In short, I encourage everyone to participate in these two venues! --Xover (talk) 09:21, 5 October 2019 (UTC)

Copyright and deletion discussions needing community input in November 2019

The following copyright discussions and proposed deletions discussions have been open for more than 14 days, and with more than 14 days since the last comments, without a clear consensus having emerged. This is typically (but not always) because the issue is not clear cut or revolves around either interpretation of policy, personal preference within the scope afforded by policy, or other judgement calls (possibly in the face of imperfect information). In order to resolve these discussions it would be valuable with wider input from the community.

Copyright discussions require some understanding of copyright and our copyright policy, but often the sticking points are not intricate questions of law so one need not be an intellectual property lawyer to provide valuable input (most actual copyright questions are clear cut, so it's usually not these that linger). For other discussions it is simply the low number of participants that makes determining a consensus challenging, and so any further input on the matter would be helpful. In some cases, even "I have no opinion on this matter" would be helpful in that it tells us that this is a question the community is comfortable letting the generally low number of participants in such discussions decide.

Copyright discussions

Proposed deletions

Note that while these are discussions that have lingered the longest without resolution, all discussions on these pages would benefit from wider input. Even if you just agree with everyone else on an obvious case, noting your agreement documents and makes obvious that fact in a way the absence of comments does not. The same reasoning applies for noting your dissent even if everyone else has voted otherwise: it is good to document that a decision was not unanimous.

In short, I encourage everyone to participate in these two venues! --Xover (talk) 09:21, 5 October 2019 (UTC)

Tech News: 2019-41

Latest tech news from the Wikimedia technical community. Please tell other users about these changes. Not all changes will affect you. Translations are available.

Recent changes

The abuse filter function now has a faster parser. This is to shorten the waiting time when you save an edit. [1]

Problems

There is a problem in the visual editor when you copy or delete text with footnotes. It will be fixed soon. [2]

Changes later this week

The new version of MediaWiki will be on test wikis and MediaWiki.org from 8 October. It will be on non-Wikipedia wikis and some Wikipedias from 9 October. It will be on all wikis from 10 October (calendar).

Meetings

You can join the technical advice meeting on IRC. During the meeting, volunteer developers can ask for advice. The meeting will be on 9 October at 15:00 (UTC). See how to join.

Future changes

The Community Wishlist Survey has a new format. It will focus on wikis that typically get less support. It will probably go back to the normal format next year. It is not decided exactly how it will work this year. You can leave feedback.
The URL of the Wikimedia wiki main pages could be changed. This is because the current URLs cause several problems. For example https://www.wikidata.org/wiki/Wikidata:Main_Page would be https://www.wikidata.org/ instead. You can tell the developers if this would cause problems for your wiki.
There is a new technical community newsletter. You can read more about the work of Wikimedia's technical community. Subscribe to get the information in the future.
Outreachy is an internship program for groups who are underrepresented in free and open-source software. There are seven Wikimedia projects about coding, documentation and quality assurance in the next round. Persons who fit the criteria can apply. The last day to apply is 5 November.

Tech news prepared by Tech News writers and posted by bot • Contribute • Translate • Get help • Give feedback • Subscribe or unsubscribe.

15:34, 7 October 2019 (UTC)

Request for comment: Are drop by copy and pastes still in scope?

When an IP address decides to copy and paste a text from an unknown source, eg. as was done at Turandot, in this time do we still consider these in scope per Wikisource:What Wikisource includes. To me, I think that we are past the point of just being an ugly text paste factory, and the work that it entails for others to curate and manage texts that are of unknown sources, and unable to be proofread further. If it was a registered editor, then I would normally follow up and see what is possible, though IP addresses are just problematic. — billinghurst sDrewth 20:58, 8 October 2019 (UTC)

If we were to consider these out of scope, where would you draw the line between copydump and legitimate text? Would you consider any text added by an IP without a clearly identified source to be out of scope? —Beleg Tâl (talk) 21:03, 8 October 2019 (UTC)

I wonder if it would be possible / beneficial to implement something like Commons' Upload Wizard, to push new users towards adding texts properly (i.e. with scans), while still leaving the regular methods available to those who know where to look, or take the time to find it. It is not foolproof but would possibly cut down on this type of drive by edits. —Beleg Tâl (talk) 21:08, 8 October 2019 (UTC)

here is a scan (does not appear to match) https://archive.org/details/turandotprincess00volluoft -- it’s unclear to me if a wizard will push this editor. don’t know if his periodic dumps rise to the level of nuisance, but yrmv. we could use some FAQ on boarding for new editors, and landing page and a welcome wagon. Slowking4 ⚔ Rama's revenge 02:25, 9 October 2019 (UTC)

@Slowking4: I am looking at a reasonable means for an administrator to look at a work and call it irretrievable, ugly high maintenance. I am not necessarily looking to be hard-arsed belligerent about works that we can suitably manage or retrieve, or where we can have a conversation with the contributor. I am looking at Turandot should give you an indicator of no source, and paste factory. If we can have a fast manage section within {{WS:PD]] rather than an indeterminate, laborious consideration, then I am fine with that. I just want a way to better cleanse. — billinghurst sDrewth 10:01, 9 October 2019 (UTC)

Hmm. I absolutely agree with Billinghurst that our standards should be higher than just accepting any old copydump absent an explicit policy reason that prevents us from hosting it. In fact, I think our standards should be higher in any number of ways, but that's outside the scope of this discussion.
Factors that weigh in favour of keeping something are: scan available, index in place, source clearly provided, plausible license tagging, at least minimal formatting, and posted by a registered user (vs. an IP). Consequently, things that weigh towards not accepting a text are absence of scan and index, no clearly discernible source, no license tagging or incorrect or implausible license tags, and lack of Wiki formatting especially actual incorrect formatting (e.g. the big grey boxes visible in Turandot), and posted by an IP or a user with no or few other contribs.
I think there should be some room to exercise judgement on these, weighing the totality of the above factors, and direct policy grounds for rejecting them (presumably a speedy criteria). Anything that has a reasonable chance of being improved to current standards should heavily favour being accepted; so, e.g., anything with a scan and Index posted by a registered user would be highly unlikely to be rejected on these grounds even if the text has multiple other problems. On the flip side, something with no formatting and lots of problems, with no or unclear source, missing or implausible license, and not posted by a registered user would be very likely to be rejected.
For something like this we should probably have an explicit "challenge + review" type process, where the work in question is automatically restored and moved to full community discussion at WS:PD.
I don't think WS:WWI is a particularly good policy hook to hang this on as a lot of crappy cut&paste jobs would be in scope if cleaned up. Better thus to have a speedy criteria with explicit built-in community review when challenged. If the uploader actually bothers initiating a challenge that fact on its own weighs in the work's favour (demonstrates interest in and willingness to work within the parameters of the project) the same way being a registered user does. --Xover (talk) 11:17, 9 October 2019 (UTC)

yeah. agree with assessment. but unclear about action plan. i would suggest an assessment, triage, put into maintenance category for further work, taskflow. it is unclear to me that WWI with a deletion task is preferable to "included but ignored / not supported / worked on" -- Slowking4 ⚔ Rama's revenge 13:01, 9 October 2019 (UTC)

I think a reasonable standard would be: if no one can figure out the source of a text dump, we delete it. Otherwise, no verification of it will ever be possible. Kaldari (talk) 21:29, 13 October 2019 (UTC)

Tech News: 2019-42

Latest tech news from the Wikimedia technical community. Please tell other users about these changes. Not all changes will affect you. Translations are available.

Recent changes

You can now use more advanced editing tools on the mobile web. You can turn them on and off in your preferences in the mobile version. [3]

Changes later this week

The new version of MediaWiki will be on test wikis and MediaWiki.org from 15 October. It will be on non-Wikipedia wikis and some Wikipedias from 16 October. It will be on all wikis from 17 October (calendar).

Meetings

You can join the technical advice meeting on IRC. During the meeting, volunteer developers can ask for advice. The meeting will be on 16 October at 15:00 (UTC). See how to join.

Future changes

Internet Explorer 6 and 7 are no longer supported. This means the browsers might start looking a bit weird. They will not get security support. You can't read Wikimedia wikis in Internet Explorer on Windows XP or Windows versions that are older than Windows XP. This is because almost no one uses the browsers anymore. Supporting them made the wikis less secure for everyone else. [4]
In the future section headings might have a share link. This is to make it easier to link to the section. You can read more and discuss.

Tech news prepared by Tech News writers and posted by bot • Contribute • Translate • Get help • Give feedback • Subscribe or unsubscribe.

23:32, 14 October 2019 (UTC)

By the way, and unrelated to this particular "Tech News" issue, but still possibly relevant to be aware of is phab:T234576. The WMF have recently removed editToken from mw.user.tokens, and the replacement is csrfToken. This affects all Gadgets and user scripts that still use the former, and will show up as either an error message (in scripts with well-written error handling) or a silent failure when a script attempts to change a wiki page. In almost all cases the fix should be simply replacing the former token with the latter. --Xover (talk) 06:18, 15 October 2019 (UTC)

Feedback wanted on Desktop Improvements project

Please help translate to your language

Hello. The Readers Web team at the WMF will work on some improvements to the desktop interface over the next couple of years. The goal is to increase usability without removing any functionality. We have been inspired by changes made by volunteers, but that currently only exist as local gadgets and user scripts, prototypes, and volunteer-led skins. We would like to begin the process of bringing some of these changes into the default experience on all Wikimedia projects.

We are currently in the research stage of this project and are looking for ideas for improvements, as well as feedback on our current ideas and mockups. So far, we have performed interviews with community members at Wikimania. We have gathered lists of previous volunteer and WMF work in this area. We are examining possible technical approaches for such changes.

We would like individual feedback on the following:

Identifying focus areas for the project we have not yet discovered
Expanding the list of existing gadgets and user scripts that are related to providing a better desktop experience. If you can think of some of these from your wiki, please let us know
Feedback on the ideas and mockups we have collected so far

We would also like to gather a list of wikis that would be interested in being test wikis for this project - these wikis would be the first to receive the updates once we’re ready to start building.

When giving feedback, please consider the following goals of the project:

Make it easier for readers to focus on the content
Provide easier access to everyday actions (e.g. search, language switching, editing)
Put things in logical and useful places
Increase consistency in the interface with other platforms - mobile web and the apps
Eliminate clutter
Plan for future growth

As well as the following constraints:

Not touching the content - no work will be done in terms of styling templates or to the structure of page contents themselves
Not removing any functionality - things might move around, but all navigational items and other functionality currently available by default will remain
No drastic changes to the layout - we're taking an evolutionary approach to the changes and want the site to continue feeling familiar to readers and editors

Please give all feedback (in any language) at mw:Talk:Reading/Web/Desktop Improvements

After this round of feedback, we plan on building a prototype of suggested changes based on the feedback we receive. You’ll hear from us again asking for feedback on this prototype.

Thank you! Quiddity (WMF) (talk)

07:15, 16 October 2019 (UTC)

Proposed update to template:header

There has been a discussion about adding some parameters to the header to capture contributors to sections/subpages of works, and to create synonyms to have more generic parameters available. The proposed changes are at template talk:header#Action to be taken October 2019. Flagging this prior to implementation in case anyone sees problems or has major issues that should stop moving forward. — billinghurst sDrewth 21:43, 17 October 2019 (UTC)

Support --Xover (talk) 08:52, 18 October 2019 (UTC)
Support, I do miss the section translator in the template. --Jan Kameníček (talk) 17:17, 18 October 2019 (UTC)

Fully validated indexes that aren't transcluded

Here are the 36 fully validated indexes that are not properly transcluded into main space (based on bot data):

Some of these have no transcluded content at all, some of them are partially transcluded, and some of them have more fundamental problems like incomplete indexes. If anyone wants to work on them, go for it! Kaldari (talk) 18:23, 16 October 2019 (UTC)

Here's a PetScan link for future reference: https://petscan.wmflabs.org/?psid=12253885 (also dibs on Townshend) —Beleg Tâl (talk) 18:43, 16 October 2019 (UTC)

Comment Which is why I generated the petscan queries ages ago, see User:Billinghurst#Petscan_queries (note that some need to be retweeked, as Magnus's petscan v.2 made his fields case sensitive for initial character (-> category and templates), and I had hoped that he would fix that issue) and why I generally get around to transcluding these at the end of each months, though Southern Historical Society Papers is a massive job, and it is taking me forever to work through Index:Dictionary of Indian Biography.djvu. — billinghurst sDrewth 21:53, 16 October 2019 (UTC)

Index:Dictionary of Indian Biography.djvu can be mostly automated, if you are interested, feel free to ping me.Mpaa (talk) 21:51, 17 October 2019 (UTC)

Comment Esme Shepherd can be pinged on her non-transcluded works as she is currently active. — billinghurst sDrewth 21:53, 16 October 2019 (UTC)

Thank you both for your efforts in making this info more accessible. Much appreciated. -Pete (talk) 21:57, 16 October 2019 (UTC)

As I have noted on my talk page, 'Index:Adelaide Contents.pdf' was transcluded 17.5.2017. 'Index:Felicia Hemans in The Christmas Box, 1829.pdf' and 'Index:Felicia Hemans in The New Monthly Magazine Volume 34 1832.pdf' have been done today. Esme Shepherd (talk) 20:35, 19 October 2019 (UTC)

Translations and source tabs

Tracked in PhabricatorTask T53980

Why do translations not have a "Source" tab? For example: Translation:Sleeping Beauty, which is transcluded from Index:La bella durmiente del bosque.djvu. Kaldari (talk) 17:31, 15 October 2019 (UTC)

Hmm. Good question. Perhaps tpt knows? --Xover (talk) 17:43, 15 October 2019 (UTC)

It's a known issue that has been outstanding since 2013 (i.e. when the Translation namespace was created), see phab:T53980 —Beleg Tâl (talk) 17:46, 15 October 2019 (UTC)

@Kaldari: because Translate: ns. was our add-on, and it probably never got coded in. Presumably there needs to be some connection the <page> building to know that it adds the /proofreadpage_source\ tab to the Translation: ns. when it transcludes pages.

I am not a coder so don't expect me to make it. — billinghurst sDrewth 05:40, 21 October 2019 (UTC)

Duplicated proofreading status with Visual Editor

Tracked in PhabricatorTask T202200

Hi! When creating a page, without any other change (just saving the content as is), if using the Visual Editor then the proofreading status is created twice (e.g. this page). Any idea? Is it a known bug, perhaps? It happens in other Wikisources as well. Thanks! -Aleator (talk) 19:03, 15 October 2019 (UTC)

Found: phab:T202200 —Beleg Tâl (talk) 20:10, 15 October 2019 (UTC)

Not being solved for 1 year and 2 months… How typical. --Jan Kameníček (talk) 21:40, 15 October 2019 (UTC)

yes, it is apparently having a status for header and status for body; status not easy to update in VE. could add to wishlist. Slowking4 ⚔ Rama's revenge 00:21, 16 October 2019 (UTC)

It would seem that it is still physically adding text

<pagequality level="1" user="" />

to the header, whereas the text is no longer actually added any more as it was melded into the page content model. I would have thought that it could have been a fairly easy fix. I would suggest that you bang on about the problem on the phabricator ticket, and ping some of the VE developers.. — billinghurst sDrewth 05:19, 16 October 2019 (UTC)

Somewhere in gerrit:/plugins/gitiles/mediawiki/extensions/ProofreadPage/+/master/modules/ve/pageTarget/ve.init.mw.ProofreadPagePageTarget.js and looking for "quality" to see where it adds the tag to the header, and presumably we want to see where it also sits in the new content model. The version presumably in the header needs to go, and ensure that VE changes the content in the other when it edits. — billinghurst sDrewth 05:46, 21 October 2019 (UTC)

Tech News: 2019-43

Latest tech news from the Wikimedia technical community. Please tell other users about these changes. Not all changes will affect you. Translations are available.

Recent changes

The API sandbox and help pages now show more clearly when API modules are marked as internal. API modules marked as internal were probably internal before. It was easier to miss. You should look for non-internal alternatives. [5]

Problems

There is a translation tool we use on wikis with more than one language. For a few days it did not create pages for new languages when someone translated a page. The languages did not show in the language bar. This has been fixed. [6]
The history and diffs can show wrong content. This is because of a cache problem. It will soon be fixed. [7]

Changes later this week

The new version of MediaWiki will be on test wikis and MediaWiki.org from 22 October. It will be on non-Wikipedia wikis and some Wikipedias from 23 October. It will be on all wikis from 24 October (calendar).
Reference Previews will be a beta feature on all Wikipedias and some Wikivoyages. It shows you a preview of the footnote when you hover over or click on the number. It has been a beta feature on German and Arabic Wikipedia since April.

Meetings

You can join the technical advice meeting on IRC. During the meeting, volunteer developers can ask for advice. The meeting will be on 23 October. See how to join.

Tech news prepared by Tech News writers and posted by bot • Contribute • Translate • Get help • Give feedback • Subscribe or unsubscribe.

14:18, 21 October 2019 (UTC)

Projects from Community Wishlist — skates time

I am reliably informed that it would be beneficial for English Wikisource to think about and plan for projects from the community wishlist, either tidying them up, or extrapolating on them, and to maybe start thinking and planning very soon, or maybe right now. Picking the best and generating a case why they are good projects with excellent outcomes.

Previous years suggestions that have been classified as Wikisource-focused can be seen at

and of course some of the more general improvements can work for us as well.

What is it that we truly want that will make our editing lives better? What is it that will make our sites truly better? What is special about our sites that could and should be better to make us better? more integrated? better findable? Let us think how we can generate benefits, and what are the benefits, rather than just think features.

Building relationships within the broader Wikimedia can be advantageous, especially if we listen. — billinghurst sDrewth 09:43, 15 October 2019 (UTC)

m:Community Wishlist Survey 2020/Wikisource is officially open. Submit, discuss and revise proposals: October 21 to November 11, 2019

Comments

Sidenotes and Layouts. We have had discussion here and never had solutions to building sidenotes from works, especially in the migration from a portrait page, to a landscape and wide computer monitor, and also onto a small mobile device. Aligned with that is the toggled layouts that were designed in prehistoric web time, and never fully functioned with sidenotes. So I would love to see if we can have some global work done on our CSS, as it is a true weakness for us. It also flows through to how we present in mobiles, and don't have good use of scripting means to present works in a known universal means, nor easy ability to check.
So I would like for one consideration for a project is to look at the output of ProofreadPage to various devices and the compatibility of our CSS to do that well. Whether this then flows to other outputs as EPUB, and other portable document formats would be interesting. — billinghurst sDrewth 10:04, 15 October 2019 (UTC)

Visual Editor, user experience design as mentioned at Wikimania,Wikisource:News/2019-09/Wikisource_at_Wikimania#Wikisource:_Challenges_and_Opportunities visual editor came to wikisource, but is not used because the menus are based on wikipedia. a UX refresh based on some user feedback would improve functionality. Slowking4 ⚔ Rama's revenge 13:59, 15 October 2019 (UTC)
@Slowking4: Are you wanting additional menus, or the ability to swap out menus? (as a non-user of VE, I am not much use here). — billinghurst sDrewth 22:59, 21 October 2019 (UTC)

i am less interested in customizing, since i use wikitext, and it would be too many choices. i tried testing it out, but it rarely helped me do what i wanted to do. it does some things well, like "find and replace", and showing line breaks. but we need a standard menu layout, based on frequent workflows and tasks. i am suggesting that until a menu redesign is done, based on newbie workflow, there will be little take-up. a regrettable lost opportunity. Slowking4 ⚔ Rama's revenge 19:12, 22 October 2019 (UTC)

OCR tool. As it has been discussed in many places, Phe's OCR tool has been out of service for many months and there is no hope it will be repaired in some reasonable time. The other available OCR tools do not work satisfactorily: either the quality is bad or they are slow. The new tool that we need should (please add more points):
- be open for repairs and maintenance, so that the whole community did not have to rely on availability of a single volunteer
- provide good quality OCR text, comparable with the Phe's tool or better
- be quick
- be able to recognize text in columns (e. g. in magazines or newspapers)
- be able to recognize foreign characters and diacritics
- …

--Jan Kameníček (talk) 09:22, 22 October 2019 (UTC)

- Most OCR tools recognize and mark bold and italicized text. If one is being created specially for WS, have it transform that into proper quote-markup. Right now, that information is lost entirely in the output. --Levana Taylor (talk) 19:29, 22 October 2019 (UTC)

- I'm not completely thrilled about marking bold; especially in much of the text we work on, bold is more often just an OCR error or something that should be marked up in a different way than something that should actually be marked bold.--Prosfilaes (talk) 21:45, 22 October 2019 (UTC)

OCR extraction from PDFs. Existing OCR text layer is very poorly extracted from PDF files here, although other applications like Acrobat extract it much better. Interestingly, the OCR extraction improves when the file format is changed into .djvu. We need to be able to extract OCR layer in the original quality directly from PDFs. --Jan Kameníček (talk) 09:30, 22 October 2019 (UTC)

PDF/A support, as described at Phabricator. --Jan Kameníček (talk) 09:31, 22 October 2019 (UTC)

yes, please put OCR on wishlist. the backlog of "Text Layer Requested" requiring use of OCR button is long. Slowking4 ⚔ Rama's revenge 19:18, 22 October 2019 (UTC)

Curly quote templates

I’m pleased to read that typographic quotes are now allowed.

One approach for editors who don't have them on their keyboards would be to create specific templates. E.g. {{sq}} could have variants like sqs (single quotes, straight) and sqc (sq, curly).

Alternately, would there be some way to have the quotes in special subpages so that something like "Name of file.djvu/dq/begin" contains the single character “ and have a template that transcludes that? You could then switch the quote style for an entire work with just 4 edits.

Opinions?

Pelagic (talk) 12:48, 15 October 2019 (UTC)

Would having a selection of them in the editing toolbar be sufficient? Templates for basic characters are fiddly and tend to look pretty cluttered in the code. --Xover (talk) 13:40, 15 October 2019 (UTC)

Could be a user option to move them out of the "Symbols" section of Special Characters to a more prominent location. Anyone up for writing an option to allow someone to place a customized set of special characters in the main editing toolbar? On second thought, there is already a gadget for keyboard shortcuts for accented characters. Easy to add quotes to that. Levana Taylor (talk) 15:30, 15 October 2019 (UTC)

Hmm, the gadget sounds good, so I have added it in my preferences now, but I cannot find anywhere whether there are any default shortcuts for some characters or how to add new shortcuts… Is there any documentation or help page about it? --Jan Kameníček (talk) 16:56, 15 October 2019 (UTC)

We can also resurrect {{dq}} and similar templates. —Beleg Tâl (talk) 15:47, 15 October 2019 (UTC)

Adding templates sounds as though we are encouraging their use. I wasn't aware that this was the plan of the community with the change. — billinghurst sDrewth 03:29, 16 October 2019 (UTC)

I think the ship has sailed … the idea of allowing curly quotes was greeted with enthusiasm by about 8 out of 10 commenters, so those same people will be busy converting texts, and it’s all to the good to facilitate doing so neatly and completely. Levana Taylor (talk) 04:17, 16 October 2019 (UTC)

Well, the issue here was perhaps more in regards using templates, specifically, for the quote marks. I don't think that's something we should encourage in the general case (with exceptions for special cases, including those where templates are the only reasonably efficient way for a given contributor to use curly quotes). All templates clutter the text in edit mode to some degree, and here we have a multiple-character expansion (6:1) that will occur tens of times on each page. It may even be enough to be a bona fide technical problem in a long quote-heavy work when you transclude a few hundred such pages into mainspace (cf. the issues with dotted toc lines).

For preference we should look to other methods to allow contributors to enter these characters, such as toolbar buttons (but at least one contributor does not use the 2010 wikitext editor or the 2017 wikitext editor), or OS/browser/keyboard-native input methods (but some OS and keyboards make that unreasonably hard). So we may need such templates for those outlier cases, but in that case we should label them accordingly as a stop-gap measure and possibly also run a continuous bot task to substitute them for the actual characters (on enwp they run a bot that automatically substs all instances of templates where the template is tagged as "must be subst:ed" that we could probably coopt for this purpose if needed). --Xover (talk) 04:58, 16 October 2019 (UTC)

I say no to templates. If people wish to do this less preferred means, then they can learn how to use all the means to add non-keyboard characters, or to use their User: drop down on their edittools set up. — billinghurst sDrewth 05:21, 16 October 2019 (UTC)

I also say no to templates. We have worked on removing the need for character templates (such as {{ae}}), so adding them for the various quote marks would be counter to our intentions. In terms of me being the main contributor who does not use either of the wikitext editors, don't initiate something specifically for me. I recognise that I am unusual in this regard. Currently things are working for me just fine without them. If I choose to work on a book in which curly quotes have been decided on, then I will add the characters to my User set in CharInsert and enter them that way. [Of course, works that I bring in and start working on will use straight quotes only.] Beeswaxcandle (talk) 06:05, 16 October 2019 (UTC)

Well, if you don't use them then it is likely that there may be more who choose not use them. I'm not saying this should be a dealbreaker for any solution we contemplate—like very old web browsers, at a certain point you just can't keep supporting them—but it would behoove us to keep the issue in mind and cater for it when reasonably possible. We want to increase the pool of contributors and make the bar to entry lower, `cause we sure ain't spoilt for folks willing to do the work! :) --Xover (talk) 06:20, 16 October 2019 (UTC)

Huh??? To whom are you directing this remark? The whole purpose of simple quotation marks is exactly for this purpose of simple editing, and why we have the style guide set as it is. The whole damn thing has been set for simplicity. In the past few years, it is the newer users who have been hyping things up trying to have exact replicas of works. The dinosaurs have long argued "KISS" and aligned with the simpler styles. — billinghurst sDrewth 11:31, 16 October 2019 (UTC)

I was responding to Beeswaxcandle's "Don't worry about me, I'll make do" by saying he's probably not the only one with that particular setup and if we can cater to it without excessive cost then we should cater to it. Recruiting new contributors is one thing, but it's equally important not to lose existing contributors (by making it harder or more frustrating for them to contribute). And in saying that I am merely stating general principles, not advocating any particular solution. But as an example (and only an example), by having a bot that automatically subst:s quote mark templates we could have our cake and eat it too: anybody that prefers it or needs it can enter them using templates, but the bot will replace them with the actual characters within minutes so that the presence of such templates in the work is only temporary. But, again, not a proposed thing we should actually do; just an example to illustrate the sort of thing that we might want to consider when the situation warrants. --Xover (talk) 14:00, 16 October 2019 (UTC)

I agree, if a template shall be allowed, it should be clear that can be replaced any time by a bot.Mpaa (talk) 22:02, 22 October 2019 (UTC)

It's all very good to say no to templates, but quote templates such as {{dq}} and {{" '}} have been around for years and are in common use throughout the site. It is not a question of encouraging their use, but rather of tolerating their continued use. —Beleg Tâl (talk) 19:38, 22 October 2019 (UTC)

Updated Module:WikidataIB

Hi to all. I have imported the newest version of this module that enables complex connections to WD and predominantly supplies data for our infobox equivalents (headers/footers) from WD items. Since we imported the script a year ago it has almost doubled in size, which would indicate increased potential for pulling data. I also imported the documentation page, so we should hopefully be able to see what functionality differences are listed in the help page (hoping!). If someone sees something of a useful nature that allows us to do more from the documentation, then please do speak up. I have protected the page as we should be inhaling the standard, stable version from Commons/enWP rather than fiddling with it locally. — billinghurst sDrewth 03:04, 28 October 2019 (UTC)

Tech News: 2019-44

Latest tech news from the Wikimedia technical community. Please tell other users about these changes. Not all changes will affect you. Translations are available.

Recent changes

You can post proposals for the Community Wishlist Survey. The survey decides what the Community Tech team will work on. You can post proposals until 11 November. You can vote on proposals from 20 November to 2 December. This year the wishlist will focus on Wikibooks, Wiktionary, Wikiquote, Wikisource, Wikiversity, Wikispecies, Wikivoyage and Wikinews. You can read more about the format for this year.
Mobile users now have a specific design for their Watchlist. [8][9]
You can share feedback and ideas on the Desktop Improvements project. The goal is to make the interface easier to use for readers and editors without removing any functionality. The Foundation's Readers Web team will work on this over the next two years.
OOUI now allows using px (pixels) instead of em (em) for some specific cases. [10][11]

Changes later this week

The new version of MediaWiki will be on test wikis and MediaWiki.org from 29 October. It will be on non-Wikipedia wikis and some Wikipedias from 30 October. It will be on all wikis from 31 October (calendar).

Meetings

You can join the technical advice meeting on IRC. During the meeting, volunteer developers can ask for advice. The meeting will be on 30 October at 15:00 (UTC). See how to join.

Future changes

There will be no software changes from 19 December to 2 January. The first MediaWiki version next year will come the week of 6 January. [12]
Gadgets and user scripts can access variables about the current page in JavaScript. In 2015, this information was moved from global variables named wg* to mw.config. The old global variables will be removed later this year. You can know more about it and tell the developers if you want to try this out on your wiki first.

Tech news prepared by Tech News writers and posted by bot • Contribute • Translate • Get help • Give feedback • Subscribe or unsubscribe.

16:09, 28 October 2019 (UTC)

PDF to wikitext import

For PDFs with embedded text (i.e. not requiring OCR) what's the simplest way getting the text of a PDF in commons converted to wikimarkup? Apologies if I'm missing an obvious help page, but I've not been able to find it. Evolution and evolvability (talk) 05:08, 29 October 2019 (UTC)

@Evolution and evolvability: See Help:Adding texts. You create an index for the work, then proofread (transcribe and format) each page, which is eventually transcluded together into a reader-visible page in mainspace. Don't hesitate to ask if you need assistance. --Xover (talk) 06:15, 29 October 2019 (UTC)

@Xover: Thanks. I think the bit I am confused by is "The text field may be blank or it might have been automatically filled with the text of that page". If uploading a PDF equivalent to e.g. this, how much of the formatting would be automatically extracted into wikitext? Evolution and evolvability (talk) 10:34, 29 October 2019 (UTC)

@Evolution and evolvability: I'm not entirely sure that document is in scope for Wikisource (we primarily host previous published works). But in any case, what you get out depends on what you put into the PDF file's text layer. You will not normally get any formatting at all, just plain text output. The process we call "Proofreading" involves not just correcting typos (scannos) and such, but also adding wiki formatting (wikifying). For example, page 2 of your example PDF contains this text layer (line-breaks and all):

PDF text layer

Wikimedia Journals
APPLICATION TO FOUND A NEW SISTER PROJECT

Background
Public trust in Wikipedia is high, yet it has long struggled to gain reputation and engage academic/expert
communities. Similarly, the quality of much of its content is superior to other encyclopedias, yet highly
variable from page to page.
As well as the first stop for information, what if Wikimedia could also be the last stop in some cases with content considered sufficiently trustworthy to be citable? The process of independent peer review
by external experts is a foundation of robust quality-control for information. This is what we have
started to achieve with the WikiJournal project.
After hundreds of years, academic publishing is finally undergoing a rapid transformation.
The Open Access (OA) movement is revolutionising reader access to peer-reviewed research, but the
publishing cost is still out of reach for billions of people who cannot afford ‘article processing fees’,
which can be thousands of dollars for one paper.
A Wikimedia journal platform would not charge for any stage of publication, relying on volunteers and
donations to run the entire project. We have shown how the WikiJournals can draw expertise from
academic and professional communities who otherwise rarely contribute to the Wikimedia movement.

What has been done so far?
Combining academic publishing and Wikipedia has been done in several formats over the last decade.
●

●

●
●

Dual publishing: In 2008 RNA Biology began requiring authors to also write a short Wikipedia
page to accompany any article on a new RNA gene family. In 2016, Gene s tarted a similar
format.
Journal first publishing: In 2012, PLOS Computational Biology created a format where authors
write an article that is published in the journal and then copied directly to Wikipedia. They were
joined by PLOS Genetics in 2016, and PLOS ONE in 2019.
Wikipedia first publishing: In 2014, Open Medicine put the first Wikipedia article through
academic peer review and publication, requiring an article processing fee.
All of the above: Since 2014, the WikiJournal User Group has run a set of journals that specialise
in these formats, hosted within Wikiversity (more in the Proof of Principle section below).

In any case, in addition to the PDF on Commons, is not meta the most appropriate place for such a proposal in wikipage format? --Xover (talk) 11:07, 29 October 2019 (UTC)

Comment if the document is a modern document and published electronically, so not requiring proofreading, then we have always taken those documents electronically, without a scan. We just need to ensure that we have the source documentation is captured, and we confirm the licence as provided. — billinghurst sDrewth 11:12, 29 October 2019 (UTC)

Search and replace button

Tracked in PhabricatorTask T183950

In one of the proposals of the current Wishlist Survey in Meta (UI improvements on Wikisource) there is a mention of some "search and replace button" among the "Advanced" functions of the Wikitext editor. However, I failed to find it. Is it available in English Wikisource too, or do I have to switch it on somewhere, or am I just blind and cannot see it? --Jan Kameníček (talk) 12:07, 30 October 2019 (UTC)

Hm, I have just found it, but it works strange: it replaces different chunks of text than I ask for!!! --Jan Kameníček (talk) 12:13, 30 October 2019 (UTC)

the find and replace on visual editor works better. (down the page options menu next to the magic pencil) you should test it out. yrmv. Slowking4 ⚔ Rama's revenge 20:49, 30 October 2019 (UTC)

Generally, my experience with VE is very bad (it always looked like the biggest bug in MediaWiki to me :-) ), but I am inclined to give it one more chance :-) However, if the button is displayed outside VE (although the programmers did their best to make it almost invisible), it should work outside VE too… --Jan Kameníček (talk) 21:50, 30 October 2019 (UTC)

i agree about VE, and it is hidden down that menu, but it works well for me. might be worth toggling to VE just to find replace. Slowking4 ⚔ Rama's revenge 22:24, 30 October 2019 (UTC)

Index:Eight Harvard Poets.djvu

The following discussion is closed:

resolved

This work is missing two pages, which are present in this scan on HathiTrust —Beleg Tâl (talk) 14:09, 22 October 2019 (UTC)

@Beleg Tâl:

Done --Xover (talk) 15:45, 22 October 2019 (UTC)

@Xover: Thanks. Discussion is continued here: Wikisource:Bot requests#Index:Eight Harvard Poets.djvu —Beleg Tâl (talk) 15:59, 22 October 2019 (UTC)

Would anyone like to volunteer to validate the two pages that were just added? —Beleg Tâl (talk) 12:24, 23 October 2019 (UTC)

@Beleg Tâl:

Done --Xover (talk) 13:22, 23 October 2019 (UTC)

This section was archived on a request by: Xover (talk) 08:04, 3 November 2019 (UTC)

Index:Dreams and Dust, by Don Marquis.djvu

The following discussion is closed:

resolved

Would it be possible for someone to swap Page:Dreams and Dust, by Don Marquis.djvu/32 and Page:Dreams and Dust, by Don Marquis.djvu/33? They are in reverse order for some reason. (The transcluded wikitext has already been swapped.) —Beleg Tâl (talk) 20:29, 22 October 2019 (UTC)

@Beleg Tâl:

Done Incidentally, this file is a good example of why DjVu rocks: the current file separates the foreground and background into separate layers and is 1.65MB (avg. 8.1kB/page). That's 1.65 MB total for 208 pages at 1874x2888 pixels, plus the hidden text layer, plus other format metadata and overhead! Just for kicks I downloaded the scans and regenerated it—and my tooling does not separate the background and foreground—and the file was 37MB (avg. 182.2kB/page). Both files are from the exact same raw scan images and the exact same resolution. The raw JPEG 2000 image files are 53MB altogether (avg. 261kB/page). CC Kaldari: who may find the data interesting. --Xover (talk) 05:58, 23 October 2019 (UTC)

This section was archived on a request by: Xover (talk) 08:05, 3 November 2019 (UTC)

help please

The following discussion is closed:

resolved

what can i add to this project? Baozon90 (talk) 18:06, 1 October 2019 (UTC)

@Baozon90: There's a lot you can do. One thing that is simple but requires a lot of care and patience is validating pages. You can find any page in Category:Proofread, ensure that the contents match the original, and mark it as validated. Adding a new text is a great thing to do as well but is more complicated. For someone entirely new to Wikisource, I'd recommend trying something small but valuable like validation as there are a lot of pages that could use it but don't require a lot of specialized skill to review. —Justin (koavf)❤T☮C☺M☯ 18:19, 1 October 2019 (UTC)

@Baozon90: If you are particularly indecisive, and want a random page from the Proofread pages, you can try the Special:RandomInCategory page; I suggest you enter "Proofread" into the field there, if you want to follow @Koavf's suggestion of starting with validation. Dcsohl (talk) 17:14, 2 October 2019 (UTC)

Wikisource:Proofread of the Month is a good place to start. and there are maintenance categories. Category:Index Not-Proofread. if you are talking about texts, we accept PD or CC texts; it helps if they are at internet archive [13] --Slowking4 ⚔ Rama's revenge 15:56, 3 October 2019 (UTC)

@Baozon90: ^^^ what Slowking4 said about POtM is perfect for learning about our systems, our quirks, and to be supported while doing so. Once you have the basics, then is the time to branch out. — billinghurst sDrewth 02:34, 4 October 2019 (UTC)

also if the Optics work is a little math heavy for you, there is a list on talk to work on Wikisource_talk:Proofread_of_the_Month. they give a good sample of work in progress. Slowking4 ⚔ Rama's revenge 23:35, 6 October 2019 (UTC)

This section was archived on a request by: Xover (talk) 08:06, 3 November 2019 (UTC)

Broken links to scans from Index pages

The following discussion is closed:

resolved

We can usually get to a scan at Commons when we click to "djvu" or "pdf" in the Source field of an Index page. However, from time to time I come across a page where the link does not work, for example at Index:Bohemian Review, 1917–Czechoslovak Review, May 1919.djvu. How can it be fixed? --Jan Kameníček (talk) 16:18, 6 October 2019 (UTC)

I experimented a bit and found that it's because of the dash in the filename. The Index page template checks if the file exists in order to suppress redlinks. It checks using {{PAGENAMEE}}, which returns the file name "Bohemian_Review,_1917%E2%80%93Czechoslovak_Review,_May_1919.djvu". Using this filename still works: File:Bohemian_Review,_1917–Czechoslovak_Review,_May_1919.djvu - but {{#ifexist:File:Bohemian_Review,_1917%E2%80%93Czechoslovak_Review,_May_1919.djvu}} returns false. —Beleg Tâl (talk) 01:52, 7 October 2019 (UTC)

I'm not sure why {{PAGENAMEE}} is used instead of {{PAGENAME}}. A related discussion at Wikisource:Scriptorium/Archives/2011-05#Special:IndexPages seems to suggest there was a bug that needed to be worked around maybe. @Billinghurst: I think you were involved in those discussions, do you know anything about this? —Beleg Tâl (talk) 01:55, 7 October 2019 (UTC)

As is known sometimes It is quite a while and probably numbers of versions ago ... <shrug> I am surprised to see it used inside File: as that seems contrary to logic, though sometimes back in the earlier days we did what worked and who knows what testcases were done, and GOIII was not the best summary-user to understand what was being done or tested. I think that file: and media: use can progress as PAGENAME rather than PAGENAMEE. — billinghurst sDrewth 07:28, 7 October 2019 (UTC)

Done and it the Index: work identified displays, though we should be checking a wider range of index pages with unusual characters in the title to look for good test cases. — billinghurst sDrewth 07:35, 7 October 2019 (UTC)

checked the first and last pages of Category:Index Validated and those unusual lead characters display fine. — billinghurst sDrewth 07:39, 7 October 2019 (UTC)

Bleh. The documentation suggests that this should fail for all files containing the characters ', ", and &. But testing does not confirm that. It does indicate some weirdness though: Index:"I solemnly swear that I won't eat no more ice cream what's made with sugar nor no more candy what's made with sugar. Ho - NARA - 512512.jpg.

What looks like might have happened here is… Well, that MediaWiki magic words are a poorly designed mess. {{PAGENAME}} returns the page name with the characters ', ", and & HTML encoded. {{PAGENAMEE}} returns the page name with a set of characters URL encoded. But the -E variant doesn't actually do real URL encoding, just a weird MediaWiki-specific variant. For proper URL encoding you need to use {{urlencode:…}} from mw:Extension:ParserFunctions. No magic word or function will actually get you the raw page name with no encoding applied.

Back when this checking was implemented in MediaWiki:Proofreadpage index template, using {{PAGENAME}} with {{#ifexist:…}} failed. I'm not quite sure how using the -E variant actually helped, but that appears to be the reason for the change. However, scanning through the listed bugs (a horror story), it looks like (but isn't actually documented anywhere that I can find), that someone at some point implemented a workaround in {{#ifexist:…}} that actually decodes the encoding applied by {{PAGENAME}} in order to make this work.

The net result is that we are relying on a poorly designed mess of stacked special-case workarounds. Lua has functions that may or may not avoid this, but I can't determine that for certain and would require rewriting the whole template in Lua which may not be worth the effort involved (then again, maybe it is!). --Xover (talk) 11:25, 7 October 2019 (UTC)

The page that you indicated above displays fine for me, no weirdness. Guessing that when we made a change to one fo the magic words, that someone changed all, for continuity, not accounting for quirks. Personally, I just want it to work, and care less about the path—it is low exposure, and presumably low overheads in the holistic sense. — billinghurst sDrewth 11:36, 7 October 2019 (UTC)

@Billinghurst: Do you see a "Source jpg" on that Index page (just above the progress field)? It's not showing up for me, not even after purging it. --Xover (talk) 11:54, 7 October 2019 (UTC)

Yes, I see it. To note that in "olden days" that we used to have to wiki-natively insert jpg images, and I would still do so when setting up an Index: page. The use of just the (page) number is not something that I would have even tried to do. — billinghurst sDrewth 12:04, 7 October 2019 (UTC)

That's very strange, as I tested it in a separate browser and not logged in; and it's not even included in the HTML source of the page. Are you sure we're looking at the same page? However, the reason it didn't show up would seem to be that the "Scans" dropdown for this index was set to "other" rather than "jpg". When I changed that it showed up properly as expected. It was thus probably unrelated to the problem that's the topic of this thread. --Xover (talk) 12:45, 7 October 2019 (UTC)

This section was archived on a request by: Xover (talk) 08:13, 3 November 2019 (UTC)

Ongoing discussion about DjVu files at Wikimedia Commons

The following discussion is closed:

proposal was withdrawn on Commons

Hello Wikisource community,

There is an ongoing discussion about the hosting of files in the DjVu format on Wikimedia Commons over at "Commons:Village pump/Proposals#DjVu is dead. We should deprecate it for new uploads.", as this format is almost universally implemented on Wikisource I request input and feedback from the Wikisource community over there. It's best to engage in discussion before "voting" as this is not a referendum but a proposal. -- DonTrung (徵國單) (討論 🤙🏻) (方孔錢 ☯) 08:05, 8 October 2019 (UTC)

As notification about this discussion has so far only been posted here at English Wikisource (and a million thanks to Donald Trung for taking the time out to do so. Very much appreciated!), but it is of potential interest to all the different language Wikisources (at least any of them that use DjVu files; but even those that do not use DjVu have indirect stakes in the outcome), we should try to make sure they are also made aware of it. I am uncertain whether we have any established channels for this. mul:Wikisource:Scriptorium would seem to be one possible avenue, but I am uncertain to what degree the non-English projects watch that forum. Can anybody advise on this? Billinghurst perhaps? --Xover (talk) 09:42, 8 October 2019 (UTC)

1) There is the wikisource-l mailing list that would be worthwhile being pinged. 2) We could look to set up a MediaWiki message delivery distribution list at Meta where we list the Wikisource wikis, and add the pages for the respective Scriptoriums per d:Q16503 (63 entries). There may be the odd other pages at some wikis if they don't have a WS:S. — billinghurst sDrewth 10:18, 8 October 2019 (UTC)

m:MassMessage and there is list of interested users at m:Global message delivery/Targets/Wikisource Community User Group participants and m:Global message delivery/Targets/Wikisource News (en) — billinghurst sDrewth 10:21, 8 October 2019 (UTC)

MassMessage for WS:S list built m:Global message delivery/Targets/Wikisource Scriptoriums

And to note as massmessage is a right allocated to admins, I have it at Meta and can send any prepared message. To note that MM at a wiki will only send local, MM at Meta is able to send globally. — billinghurst sDrewth 10:37, 8 October 2019 (UTC)

@Billinghurst: Wouldn't Donald's original notification here work well?

Sure, I simply wish for it to be seen as the community's considered and sanctioned message, rather than mine. Whether it is notification alone, or part opinion, or consideration of consequence, or condemnation … — billinghurst sDrewth 20:17, 8 October 2019 (UTC)

Hello Wikisource community,
There is an ongoing discussion about the hosting of files in the DjVu format on Wikimedia Commons over at "Commons:Village pump/Proposals#DjVu is dead. We should deprecate it for new uploads.", as this format is almost universally implemented on Wikisource your input and feedback would be valuable over there.

Or perhaps we should specify that that this is the English Wikisource community passing it on? Perhaps by tacking on a "The English Wikisource community has been made aware of a discussion that may be of interest to your project." at the beginning? --Xover (talk) 13:25, 8 October 2019 (UTC)

I withdrew the proposal. Kaldari (talk) 05:13, 9 October 2019 (UTC)

i kinda agree with the assessment, but would prefer an action plan to make IA uploader produce pdf’s from jp2. this would sunset the djvu. -- Slowking4 ⚔ Rama's revenge 13:09, 9 October 2019 (UTC)

Such an action plan will have little value if the points raised by Xover at Commons are not solved. --Jan Kameníček (talk) 17:38, 9 October 2019 (UTC)

This section was archived on a request by: Xover (talk) 08:16, 3 November 2019 (UTC)

MediaWiki:InterWikiTransclusion.js

The following discussion is closed:

Resolved: script is now imported from global version instead of maintaining a local copy

Is there a reason we maintain our own copy of MediaWiki:InterWikiTransclusion.js instead of just using mul:MediaWiki:InterWikiTransclusion.js directly like frWS does? Our copy is a couple of years out of date and is missing functionality for section transclusion including fixes for phab:T188202. Some pages such as Lapsus Calami (Apr 1891)/Coll. Regal. are broken because our copy is missing these updates. —Beleg Tâl (talk) 15:22, 11 October 2019 (UTC)

@Beleg Tâl: I have changed the file to pull the mulWS script, please see whether it works as expected, there may be dependencies that need to be expressed. — billinghurst sDrewth 09:00, 14 October 2019 (UTC)

Not making a difference for me. — billinghurst sDrewth 09:03, 14 October 2019 (UTC)

Now it does. That damn caching of common.js. — billinghurst sDrewth 09:35, 14 October 2019 (UTC)

This section was archived on a request by: Xover (talk) 08:17, 3 November 2019 (UTC)

"Studies in Irish history, 1649-1775" available in scanned form?

Is anyone able to see a downloadable, OCR'd copy of the work "Studies in Irish history, 1649-1775" by Murray, Alice Effie b. 1877., Gwynn, Stephen Lucius 1864-1950., Mangan, H. , Wilson, Philip. , Butler, William Francis Sir 1838-1910. et al.

I see that it is at HathiTrust though no indication of whether scanned.

If it is, it would be great if someone can get it into archive.org so we can take it to Commons. Thanks. — billinghurst sDrewth 03:18, 13 October 2019 (UTC)

It says full view at the bottom, and the first scan (unusually for HathiTrust) even lets you download the PDF directly. I've got a copy of the PDF, though at over 100 MB, I've got to upload it to IA and then sideload it to Commons.--Prosfilaes (talk) 03:33, 13 October 2019 (UTC)

I can't upload it to Commons, since one of the authors died 1950, and there's three others with no death date at hand. It's too big for me to upload it to Wikisource. It's at https://archive.org/details/studiesinirishhistory16491775 , if anyone knows how to get it here.--Prosfilaes (talk) 03:42, 13 October 2019 (UTC)

@Prosfilaes: Thanks, the download links must be country specific, and as I am not operating from a VPN, it knows that I am not US-based.. I will pull it onto Wikisource once it has derived a new form. If it is still over 100MB, then we can get someone to upload it on the backend. — billinghurst sDrewth 03:47, 13 October 2019 (UTC)

Looking around more, this is the same scan as https://archive.org/details/studiesinirishhi02obri . The HathiTrust version has an update date of 2012, so there may be a rescan or two, but it ultimately came from the same source.--Prosfilaes (talk) 04:25, 13 October 2019 (UTC)

What?!? I don't even see it in a search result. Sheesh, sorry to put you to the effort. — billinghurst sDrewth 05:40, 13 October 2019 (UTC)

@Billinghurst: Ad not US-based: Try Hathi Download Helper, but the downloading times are long. --Jan Kameníček (talk) 19:30, 13 October 2019 (UTC)

┌─────────┘
@Billinghurst, @Prosfilaes, @Jan.Kamenicek: I didn't quite follow the above, but took a guess and on the chance it'd be useful I grabbed Internet Archive identifier: studiesinirishhi02obri, regenerated a new DjVu from it, and uploaded here as File:Studies in Irish History, 1649-1775 (1903).djvu with an index at Index:Studies in Irish History, 1649-1775 (1903).djvu. If it's not needed or or borked in some way then feel free to delete. --Xover (talk) 11:29, 3 November 2019 (UTC)

IA-Upload tool is down

The following discussion is closed:

tool is back up again

@Samwilson: @Tpt: FYI —Beleg Tâl (talk) 14:55, 16 October 2019 (UTC)

Given the timing I'm going to go ahead and guess that this is related the ongoing reboot of the cloud hosts on which the toolserver/toolforge runs. There was one batch last Wednesday, one batch today, and will be another one next Wednesday. From the error message it looks it might just be that the service needs to be started again. --Xover (talk) 15:38, 16 October 2019 (UTC)

@Xover, @Tpt: I've restarted the web service and it seems to be fine again now. —Sam Wilson 00:02, 17 October 2019 (UTC)

It was down for 14 hours, 45 minutes and 58 seconds. Sam Wilson 00:17, 17 October 2019 (UTC)

This section was archived on a request by: Xover (talk) 08:23, 3 November 2019 (UTC)

The source of local time digital clock display

The following discussion is closed:

resolved

I have the local time displayed in the upper right corner but the local clock link in my local Vector.js is disabled and the clock in Gadgets is not selected.

The disabled link my local Vector.js is

//mw.loader.load www.mediawiki.org/w/index.php?title=MediaWiki:Gadget-LocalLiveClock.js&action=raw&ctype=text javascript&smaxage=21600&maxage=86400');

From where does the local time display originates? — Ineuw (talk) 00:21, 27 October 2019 (UTC)

If your gadget is turned off, then it can come from the skin's js, your local common js or your global js configurations. PS. The time displayed is via your preference settings, it isn't local per se. — billinghurst sDrewth 00:48, 27 October 2019 (UTC)

Thanks I know this and checked the vector and global .js files and they were disabled. That is why I asked. It was just curiosity. — Ineuw (talk) 21:26, 27 October 2019 (UTC)

Just figured out where it comes from. When all clock scripts are disabled, in preferences\appearance, there is a local exception for the time. When selected will only display the local time. No script is needed. — Ineuw (talk) 21:32, 27 October 2019 (UTC)

This section was archived on a request by: Xover (talk) 08:23, 3 November 2019 (UTC)

Index:Carroll - Alice's Adventures in Wonderland.djvu

The following discussion is closed:

resolved

Please see Kaldari's request at WS:S/H#Need help fixing Alice's Adventures in Wonderland —Beleg Tâl (talk) 17:36, 30 October 2019 (UTC)

This section was archived on a request by: Kaldari (talk) 18:56, 11 November 2019 (UTC)

Index:Carroll - Alice's Adventures in Wonderland.djvu

The following discussion is closed:

Missing page has been incorporated into the work.

I believe I have found a copy of the missing plate from this book on flickr, by using Google image search, but I have been unable to locate the missing text page. (See the discussion on the project page.)

I have uploaded the copy of the plate that I found into the book category at commons ( Charles Robinson's illustrations of Alice's Adventures in Wonderland ) it is the one named Alice's Adventures in Wonderland - Carroll, Robinson - S205 - The whole pack rose up in the air.jpg

Could someone take a look to see if it can be used and if so inserted into the book at the appropriate point?

If it is not suitable please delete it.

Thanks Sp1nd01 (talk) 14:19, 28 October 2019 (UTC)

@Sp1nd01: Thanks! It's been incorporated into the work. Kaldari (talk) 19:16, 11 November 2019 (UTC)

This section was archived on a request by: Xover (talk) 13:56, 29 November 2019 (UTC)

KaldariBot

Bot name: KaldariBot (talk • contribs)
Bot operator: Kaldari (talk • contribs) (home wiki: en.WP)
Automatic or manually assisted: automatic
Purpose of the bot: Add {{validated}} template to validated works so that they are properly categorized and can be searched separately.
Edit period(s): Continuous for 1 day, 1-6 edits/minute
Programming language(s) (and API) used: PHP (mediawiki-api-base, botclasses)
Affected pages: The 3086 pages listed at https://github.com/kaldari/validationbot/blob/master/public_html/alphalist.php
Additional information: This bot request is per the proposal and discussion at Wikisource:Scriptorium/Archives/2019-08#Making our validated texts discoverable.

Comment if we are going to do this, what is the possibility to put the "validated" flag on the wikidata item interwiki for the respective works? To note that I am gathering that this is for situations where the Index: page has been marked as validated and there is a one-to-one relationship with a main namespace page. To note that this list does include subpages of works where the works have been uploaded as parts, some would warrant listing indepedently as validated, others, not so. [A find of <code/ shows interesting output. — billinghurst sDrewth 05:35, 31 October 2019 (UTC)

@Kaldari: I agree with billinghurst regarding subpages. I would leave them out as it might be controversial and limit entries to top level items for now.Mpaa (talk) 14:44, 10 November 2019 (UTC)

Please check your list for redirects. I have found one redirect in the list. — billinghurst sDrewth 05:39, 31 October 2019 (UTC)

I was planning to have the bot follow redirects when posting the template, but I'll just go ahead and replace any redirects with their targets in the list... Kaldari (talk) 23:03, 31 October 2019 (UTC)

@Billinghurst: I've replaced all the redirects in the list with their ultimate targets. Kaldari (talk) 19:42, 1 November 2019 (UTC)

Adding the flag in Wikidata should be fairly easy to do if the wikibase-api library supports it. If not, I'll need to dig into the actual Wikibase API, which might be complicated. Kaldari (talk) 00:12, 1 November 2019 (UTC)

It looks like wikibase-api does support setting badges. However, I think it would be best to add the badges after all the pages have been templated, categorized, and reviewed for accuracy. My list is mainly based on following title links from the Indexes. However, I've noticed that use of the title field varies considerably. Some link to disambiguation pages, some link to multiple pages, and some don't have links at all. I've tried to go through all the ones that are obviously problematic and fix them by hand, but I imagine I will have missed some. Once the pages are added to Category:Validated texts (by the template), it will be easier to review them all, as I can just open them in new tabs from the category page. Once they are reviewed, anyone could write a script to badge all the pages in the category. Kaldari (talk) 19:41, 1 November 2019 (UTC)

I think that we should have the header module pull the Validated status from Wikidata and display the badge that way, but I support having a bot ensure that the status is correct on Wikidata. —Beleg Tâl (talk) 21:55, 1 November 2019 (UTC)

@Beleg Tâl: I think that's a good idea in theory, but there are some practical problems. Few Wikisource editors bother to create or link Wikidata items to their works. Of the first 5 works in my list, only 2 were linked to Wikisource. If people aren't even linking to Wikidata consistently, I think there's a vanishingly small chance that they will try to keep the Wikidata badges up to date. I imagine that people will just start adding their works to Category:Validated texts manually, as it won't be very intuitive that you have to set a badge in Wikidata (a very obscure feature) in order to add the work to the Category on Wikisource. Plus I don't really see what we gain by having the status tracked on Wikidata rather than in Wikisource directly. Kaldari (talk) 01:33, 2 November 2019 (UTC)

@Kaldari: users will not add Wikidata badges manually, but they will not add Category:Validated texts manually either (I certainly won't). We'd need a bot either way, so we may as well have the bot do it "properly" i.e. by leveraging Wikidata (and thus preventing duplicate data). If we need a bot to create the Wikidata item in the first place, then we should look into that also. —Beleg Tâl (talk) 14:54, 2 November 2019 (UTC)

If I am not wrong, 922 items are missing wikidata item. I tried to pull Category:Validated texts from wikidata badge but I did not succeed, I could only pull Category:Validated. Anyone knows how?Mpaa (talk) 22:21, 5 November 2019 (UTC)

@Mpaa: The idea that I put forward, would be to have the header module check the text's associated data item, and if the validated badge is present, then it would add Category:Validated texts to the header (similar to how Category:Works with non-existent author pages is added by the header based on the presence of an associated author page, though this uses #ifexist instead of a Wikidata query) —Beleg Tâl (talk) 00:48, 6 November 2019 (UTC)

I got the idea, I am wondering if someone knows how to pull the badge for a sidlelink of a wikidata item, with the current available modules. If not, or if this is not supported, this doesnt seem a good way-forward at the moment, until this point is cleared.Mpaa (talk) 22:18, 8 November 2019 (UTC)

@Mpaa: Module:Edition contains code for retrieving the badge of the sitelink, though it doesn't do anything useful with it. —Beleg Tâl (talk) 15:29, 9 November 2019 (UTC)

@Beleg Tâl:, seems there are a few more steps to be done, like modify Module:Edition to get the badge ID or the category we want to associate to it.Mpaa (talk) 18:21, 9 November 2019 (UTC)

@Mpaa: it looks like w:Module:Wd does it properly, so we could just import that module. —Beleg Tâl (talk) 13:58, 10 November 2019 (UTC)

Sounds good to me. I am not very familiar with importing pages (I am uncertain if this needs to be flagged or not in this case: Include all templates), if someone volunteers better so, otherwise I will give it a try.

Update: As it seems the desired route is to record the information in Wikidata, I will need to write some more code: first to collect all the associated Wikidata items (unless Mpaa has already done this) and then to add the badges in Wikidata. Unfortunately, I'll be at the Wikimedia Technical Conference all next week, but hopefully I can start working on it afterwards if everyone agrees this is the best solution. Kaldari (talk) 20:08, 8 November 2019 (UTC)

If needed, I can generate a list, and also create the missing WD items. Mpaa (talk) 21:21, 11 November 2019 (UTC)

@EncycloPetey, @Billinghurst, @Mpaa, @Beleg Tâl: The first step of this process is now complete. We are now automatically pulling badge data from Wikidata and displaying it on Wikisource via the {{header}} template. We are also now automatically categorizing works based on these badges, for example, Category:Validated texts, Category:Proofread texts, etc. The next step is to add badges on Wikidata for texts that are missing them (via a bot). Before we do this, however, two questions need to be answered:

Should "featured" text status replace the "validated" status or exist along-side it? In other words, is a featured text both "featured" and "validated", or is it just "featured" (which implies that it is also validated, proofread, etc.)?
Should "digital document" status exist along-side other text statuses or replace them? In other words, can a text be both a "digital document" and "validated" (even though in theory a digital document shouldn't need proofreading and validation)?

Once these questions are settled, I can move ahead with the bot work. Kaldari (talk) 22:38, 3 December 2019 (UTC)

Typically, I have replaced the Validated badge with the Featured badge when marking Featured texts, but we could decide to retain both. I'm satisfied either way on that issue. The "digital document" badge refers only to the original, and applies only to works that were originally digital. It should never appear on works that were scanned from physical objects. Validation applies only to the process of verification, so there is no reason not to display both badges when both apply. However, I personally think it is inappropriate to use "digital document" as a badge, because the badges are meant to show quality of the work, and "digital document" is a statement about the source of a document rather than its status or quality on Wikisource. Even digital documents can require adapting to display correctly on Wikisource. --EncycloPetey (talk) 22:52, 3 December 2019 (UTC)

So it sounds like you would be OK with the bot just adding validated badges without regard to whether the work already has a featured or digital document badge. If no one else has an opinion on the matter, I would be happy to implement it that way. Kaldari (talk) 16:07, 11 December 2019 (UTC)

Update to NopInserter Gadget

The following discussion is closed:

There is a clear absence of community support for this change, though the lack of even a single objection suggests this may be due to apathy or lack of interest rather than any perceived or actual problem with the proposal. In future, the community is strongly urged to express their preference explicitly in order to enable consensus-based processes to function.

A while back, while debugging an unrelated issue, I found a bug in MediaWiki:Gadget-NopInserter.js that prevented it from displaying the intended visual indication of its operation. I also found that at the time of implementation there had been some differing preferences for what type of visual indicator be used and when prompting for confirmation was appropriate. I have therefore created an updated version in User:Xover/Gadget-NopInserter.js that fixes the bug and adds configuration options for whether to confirm addition of a {{nop}}, the style of visual indicator, and the duration of the indicator effect. To try it out you can add the following to your common.js (but disable the site-wide gadget in your preferences first!):

mw.config.set('userjs-nopinserter', {
	dontConfirmNopAddition: true,
	notificationStyle: "highlight",
	notificationTimeout: 1000
});

mw.loader.load('//en.wikisource.org/w/index.php?title=User:Xover/Gadget-NopInserter.js&action=raw&ctype=text/javascript');

It also fixes the bug that prevented the site-wide gadget from actually showing the outline based highlight it was supposed to. And for good measure I added support for a notificationStyle using mediawiki's bubble notifications (set notificationStyle: "message" to try it out). The weird double-negative construction of "dontConfirmNopAddition" is just because I've preserved the default behaviour of the site-wide gadget. If you remove everything except the mw.loader.load line (no setting of options) you will get the old default behaviour with just the bugfix.

The changes can be seen in this diff.

The changed version has had some limited testing and seems ready for wider testing. I therefore propose that we update MediaWiki:Gadget-NopInserter.js with this version. Note that since we do not have interface administrators locally, I will have to request this edit from the Stewards at meta, and they will require a community discussion to verify that this is indeed a change in line with community consensus. It would therefore be very helpful if as many as possible indicated whether or not you support this proposal. --Xover (talk) 12:37, 6 October 2019 (UTC)

I think local bureaucrats can set the "Interface administrators" bit.Mpaa (talk) 14:18, 6 October 2019 (UTC)

Bureaucrats have the technical ability to flip that bit, yes. But by WMF Legal-imposed policy it requires 2FA, and so can't just be assigned ad hoc like other local permissions. And since we have no permanent interface admins, nor any "list of people willing and able to make interface admin-edits", we don't actually have any functioning local way to request such changes; unless you yourself happen to have 2FA enabled for other reasons. Thus, asking the Stewards at Meta is actually the easiest option for getting such changes made currently. --Xover (talk) 05:55, 7 October 2019 (UTC)

OK, just a bit weird that it is 'technically' possible but not 'legally' possible without 2FA. If they want to be on the safe side, they should not allow without 2FA.

Support Anyhow, I am fine with the proposal.Mpaa (talk) 19:37, 7 October 2019 (UTC)

Yeah, the 2FA requirement is kinda dumb to begin with (not that it helps that we don't have a functioning local Interface Admin policy). In any case, this thread, so far, does not demonstrate community support for the proposed change (absence of objections is not the same as support), so it appears this change will not be implemented. Note that if the community's reticence should happen to be about the other changes, I can redo this patch to only fix the (7 years old) bug. In the mean time, anyone that wishes may of course use the copy in my userspace using the syntax described above, but absent any indications of interest I probably will not be actively maintaining that copy. --Xover (talk) 13:39, 5 November 2019 (UTC)

This section was archived on a request by: Xover (talk) 08:18, 14 December 2019 (UTC)